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Chapter  1 


INTRODUCTION 

Over v i ew 

The  Defense  Acquisition  Regulation  (DAR)  requires 
that  price  or  cost  analysis  be  performed  on  all  negotiated 
contracting  actions  (22:3:122).  While  this  analysis  is  the 
ultimate  responsibility  of  the  Contracting  Officer*  (CO), it 
can  be  performed  by  either  the  CO  or  by  an  independent 
staff.  The  DAR  does  not  specify  the  circumstances  under 
which  pricing  staff  assistance  is  required,  but  does  pro¬ 
vide  general  guidelines. 

According  to  the  DAR.  pricing  staff  support  should 
be  obtained  "when  complex  techniques  are  indicated 
[22:3:111]."  The  DAR  does  not  define  complex  techniques. 

A  related  directive,  the  Defense  Acquisition  Regulation 
Manual  (DARM)  No.  1  while  suggesting  that  pricing  support 
be  obtained  on  actions  "involving  costly,  technically 
complex,  or  unique  requirements  [23:1 A2 ] " .  does  not  define 
complexity  either.  Possibly  because  complexity  is  so 


For  the  purposes  of  this  thesis,  the  term 
"Contracting  Officer1’  refers  to  the  person  in  the  contrac¬ 
ting  office  having  responsibility  for  the  effort.  It 
could  be  either  the  Principle  Contracting  Officer  (PCO)  or 
the  buyer . 
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subjective,  in  actual  practice,  cost  is  the  only  factor 
considered  by  the  contracting  and  pricing  offices  (6:23). 


The  directives  delegate  the  responsibility  for 
implementing  the  general  guidelines  to  the  major  commands. 
Within  both  the  Air  Force  Systems  Command  (AFSC)  and  the 
Air  Force  Logistics  Command  (AFLC) .  the  two  commands  with 
major  Air  Force  acquisition  responsibility,  every  contrac¬ 
ting  directorate  has  established  a  dollar  threshold  as  the 
sole  criterion  for  determining  if  pricing  staff  action  is 
required  (6:23) . 

Problem  Statement 

Current  implementation  of  contracting  directives 
establishes  dollar  value  as  the  sole  criterion  for  pricing 
staff  analysis  of  a  contracting  action.  This  singular 
criterion  may  be  causing  unnecessary  expenditure  of  effort 
by  the  pricing  staff  on  actions  above  the  threshold  which 
could  be  priced  by  the  CO  alone.  And,  at  the  same  time, 
the  theshold  may  be  eliminating  from  the  pricing  staff's 
attention  actions  below  the  threshold  which  require  pricing 
expertise  beyond  that  of  the  CO. 

Background 

The  purpose  of  the  defense  acquisition  system  is 
to  develop  and  supply  the  weapons,  equipment,  and  services 
required  to  meet  United  States  national  defense  objectives. 
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Within  the  Air  Force,  this  responsibility  is  carried  out 
primarily  by  the  two  acquisition  commands:  AFSC  and  AFLC. 
The  AFSC  is  responsible  for  the  conception,  design,  devel¬ 
opment,  and  production  of  Air  Force  systems.  In  terms  of 
acquisition  functions.  AFSC  responsibility  includes  the 
purchasing  of  design  studies,  research  and  development, 
prototypes,  engineering  development,  and  initial  production. 
After  a  system  has  reached  the  production  stage,  responsi¬ 
bility  is  shifted  to  AFLC.  In  acquisition  terms.  AFLC  is 
responsible  for  purchasing  the  necessary  support  to  operate 
and  maintain  the  system  throughout  its  system  life. 

In  both  commands,  the  acquisition  process  is 
basically  the  same.  This  process  involves  the  steps  of 
purchase  request  review,  solicitation,  evaluation,  and 
award.  Within  the  Department  of  Defense  (DOD),  there  are 
two  methods  for  carrying  out  this  acquisition  process: 
formal  advertising  and  negotiation.  Formal  advertising  is 
the  method  whereby  bids  are  solicited  and  received  from 
two  or  more  offerers,  bidding  independently,  and  whereby 
the  award  will  be  made  to  the  lowest  responsible,  respon¬ 
sive  bidJer  (22:2:2).  While  this  is  the  preferred  method 
of  contracting,  only  12  percent  of  over  $18  billion  awarded 
for  Fiscal  Year  (FY)  78  by  the  Air  Force  was  awarded 
through  formal  advertising  (5:32-37). 
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As  a  result,  approximately  $16  billion  spent  by  the 
Air  Force  in  FY  78  were  awarded  as  a  result  of  negotiation. 
Under  negotiation,  which  can  be  either  sole  source  or  com¬ 
petitive,  award  is  not  always  made  to  the  lowest  offerer, 
but  rather  to  the  offerer  whose  offer  is  "most  advantageous; 
price  and  other  factors  considered  [22:3:65A)."  Thi6  places 
a  requirement  on  the  contracting  office  to  evaluate,  among 
other  items,  the  price  of  the  offer.  This  is  done  through 
price  and  cost  analysis. 

Price  analysis  is  the  evaluation  of  the  total  price 
(cost  and  profit/fee)  of  an  item  without  the  analysis  of 
individual  elements  of  cost  (23: 3 A 1 ) .  This  analysis  is 
basically  a  comparative  technique  which  looks  at  only 
bottom  line  prices  and  which  involves  relatively  simple 
analytical  methods.  Price  analysis  is  performed  either 
alone  or  in  conjunction  with  cost  analysis,  on  every  nego¬ 
tiated  action.  Whenever  price  analysis  alone  is  sufficient 
to  determine  the  reasonableness  of  a  price,  the  CO  will 
generally  perform  the  analysis  without  supporting  action 
from  the  pricing  staff.  However,  in  cases  involving  both 
cost  and  price  analysis,  the  CO  may  require  assistance  from 
the  pricing  staff. 

Cost  analysis,  on  the  other  hand,  is  much  more 
detailed,  involving  an  element  by  element  examination  of 
cost,  and  is  required  on  every  negotiated  procurement  above 
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$100,000.00  (23: 3B 1 ) .  It  includes  verification  of  cost 
data,  evaluation  of  specific  elements  of  cost,  and  projec¬ 
tion  of  these  data  to  determine  the  effect  on  price  of 
such  factors  as: 

1.  necessity  of  costs, 

2.  r easonableness  of  the  amounts  of  specific 

costs , 

3.  allowances  for  contingencies. 

4.  basis  used  for  allocation  of  indirect  costs, 

and 

5.  appropr iateness  of  allocation  of  specific 
indirect  costs  to  a  particular  action  (22:3:  122)  . 

Cost  analysis  is  considerably  more  complex  and  time 
consuming  than  price  analysis.  Frequently  it  involves 
highly  sophisticated  analytical  techniques  such  as  para¬ 
metric  cost  estimating,  regression  analysis,  and  trends 
studies  This  type  of  analysis  generally  requires 
specialized  knowledge  and  expertise  which  are  usually  found 
only  in  the  pricing  office. 

Every  major  contracting  directorate  within  AFSC  and 
AFLC  has  a  separate  pricing  office.  The  pricing  office, 
which  in  spite  of  its  name  performs  both  price  and  cost 
analysis,  serves  as  a  staff  function  and  supports  all  of 
the  individual  contracting  offices  within  the  directorate. 
For  instance,  the  staff  pricing  office  of  the  Directorate 
of  Research  and  Development  Contracting.  Aeronautical 
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Systems  Division  (ASD).  Ur igh t-Pat terson  AFB,  OH  provides 
pricing  assistance  for  all  of  the  buying  divisions  within 
the  directorate  (see  Appendix  A). 

The  primary  function  of  the  pricing  office  is  to 
assist  and  support  the  CO  in  gathering,  assimilating,  and 
evaluating  cost  data,  in  establishing  the  government  nego¬ 
tiation  objectives,  and  in  conducting  negotiations  (23: 1 A3 ) . 
Specific  tasks  of  the  pricing  office  include  price  analysis, 
cost  analysis,  fact-finding,  negotiation,  and  documentation 
(6:63).  The  pricing  office  is  a  staff  agency  reporting 
directly  to  the  Director  of  Contracting  and  is  autonomous 
from  the  buying  divisions. 

Price  analysts  are  specifically  trained  in  their 
tasks  of  price  and  cost  analysis.  These  are  the  only  func¬ 
tions  performed  and  therefore  analysts  are  not  involved  in 
other  contracting  duties  as  are  COs .  As  a  result,  pricing 
personnel  are  considered  to  be  "pricing  experts"  who  possess 
technical  competence  and  expertise  beyond  that  of  a  CO. 

While  the  function  of  the  pricing  office  is  to 
provide  support  to  the  CO,  it  was  not  intended  that  support 
be  provided  on  every  action  requiring  analysis.  This  is 
evidenced  by  the  job  description  provision  that  20  percent 
of  a  journeyman  CO's  time  be  spent  on  pricing  (2.19).  It 
appears  then,  that  the  role  of  the  pricing  office  is  to 
provide  support  in  only  those  instances  requiring  expertise 
beyond  that  of  the  CO. 
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Currently,  contracting  offices  are  operationally 
defining  contracts  which  require  special  expertise  as 
those  that  are  above  a  specified  dollar  threshold.  Each 
contracting  office  has  established  its  own  threshold  based 
upon  dollar  value  of  the  actions  handled  by  the  office  and 
by  the  workload  level.  The  threshold  varies  significantly 
from  office  to  office.  For  instance  in  AFSC,  the  threshold 
ranges  from  a  low  of  $200,000.00  for  Electronic  Systems 
Division  (ESD)  to  a  high  of  $2,000,000.00  for  Aeronautical 
Systems  Division  (ASD)  (1:1:5;  7:1-3).  In  AFLC  all  offices 
have  a  threshold  of  $100,000.00  (18). 

The  dollar  threshold  provides  a  very  convenient  and 
efficient  criterion.  It  is  precise,  unambiguous,  and  easily 

understood.  It  is  extremely  simple  to  apply  and  can  be 

2 

readily  adapted  to  varying  conditions.  However,  the  dollar 
threshold  is  an  arbitrary  measurement,  and  as  such  may  not 
reflect  complexity  and  uniqueness,  factors  which  the  direc¬ 
tives  suggest  should  be  used  by  the  CO  in  determining  if 
the  special  expertise  of  the  pricing  staff  is  required. 

The  purpose  of  this  study,  therefore,  was  to  inves¬ 
tigate  the  effectiveness  <  1  the  dollar  threshold  and 


*■  The  AFSC  and  AFLC  originally  established  their 
threshold  at  $100,000.00  (the  level  at  which  certified 
cost  and  pricing  data  are  required).  AFSC  offices  have 
frequently  adjusted  their  thresholds  to  reflect  increasing 
workloads  and  inflation.  AFLC  offices,  however,  have  not 
made  any  adjustments. 
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feasibility  of  alternate  criteria.  This  investigation, 
based  upon  a  number  of  assumptions,  attempted  to  meet  the 
listed  objectives  by  answering  five  questions. 

0  ~~  " 

f  '  •y 

Assumpt ions 

The  effective  use  of  resources  is  a  problem  which 
is  often  linked  to  manpower  allocation.  Whether  or  not 
resources  are  being  used  effectively  usually  is  not 
recognized  until  a  manpower  or  workload  problem  exists. 

In  recent  years,  a  serious  workload  problem  has  surfaced 
within  the  pricing  staff  offices  (13).  This  problem  is 
most  likely  due  to  manpower  shortages  as  well  as  other 
factors,  such  as.  ineffective  use  of  resources.  While  it 
is  recognized  that  a  manpower  shortage  may  exist  and  may 
be  related  to  the  research  problem,  it  is  not  considered 
to  be  an  issue  that  can  be  resolved  since  additional  man¬ 
power  authorizations  will  not  be  approved  by  the  United 
States  Air  Force  (USAF)  (13).  Therefore,  this  study  assumes 
that  the  present  manpower  situation  will  not  change  and 
looks  for  ways  to  increase  the  effectiveness  of  the  utili¬ 
zation  of  the  existing  resources. 

Research  Objectives 

The  primary  objectives  of  this  research  effort  are 
to: 
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1.  Determine  what  criteria  are  perceived  by  the 
pricing  community  to  influence  the  need  for  pricing  staff 
action . 


2.  Use  the  new  criteria  to  develop  a  decision 
model  which  will  predict  when  pricing  staff  action  is 
required. 

3.  Test  the  use  of  the  model  to  determine  if  its 
use  results  in  different  decisions  than  the  use  of  the 
dollar  threshold. 


Research  Questions 

1.  How  can  the  current  method  of  determining 

the  need  for  pricing  staff  action  be  improved  with  respect 
to  more  efficient  and  effective  utilization  of  pricing 
resources? 

2.  What  factors  are  considered  important  in  the 
pricing  decision? 

3.  What  skills  do  pricing  staff  personnel  possess 
that  are  unique  from  those  of  the  CO? 

4.  What  constitutes  "complexity"  as  referred  to 
in  the  DAR  and  how  can  it  be  measured? 

5.  Will  the  use  of  criteria  other  than  dollar 
value  result  in  a  more  effective  use  of  pricing  resources? 

Scope  and  Limitations 

This  study  was  limited  to  pricing  staff  offices 
within  AFSC  and  AFLC.  Since  these  commands  constitute  the 
major  acquisition  centers  in  the  Air  Force,  this  limitation 
is  considered  appropriate. 

Just i f  icat ion 

In  recent  years  the  number  of  pricing  actions  on 
an  Air  Force  wide  basis  has  been  steadily  increasing  while 
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Che  workforce  has  been  decreasing  (6:17,59).  As  a  result, 
pricing  offices  have  been  unable  to  handle  the  workload 
and  have  been  often  forced  to  return  actions  to  the  COs 
for  analysis.  Although  no  complete  records  are  available 
which  indicate  the  number  of  actions  returned  to  Che  COs 
for  analysis,  the  magnitude  of  the  problem  is  evidenced  by 
the  continual  raising  of  the  thresholds  within  AFSC  offices. 
The  higher  thresholds  reduce  the  number  of  actions  which 
are  forwarded.  As  a  result  many  actions  which  could  benefit 
from  the  pricing  staff’s  expertise  are  not  being  forwarded. 

In  addition,  Headquarters  United  States  Air  Force 
(HQ  USAF)  has  expressed  concern  over  the  efficiency  of 
analysis  being  performed  by  both  the  pricing  offices  and 
the  COs.  Major  Grady  Jacobs,  Contract  Pricing  Air  Staff 
Officer,  expressed  that  same  concern  and  requested  that  a 
study  of  the  criteria  for  pricing  action  be  conducted  (13). 
Operating  level  personnel  have  also  expressed  a  concern 
that  the  current  criterion  is  not  adequate.  Lt  Col  Richard 
Goven.  ASD  Pricing,  and  Mr.  Harry  Curtner,  ESD  Pricing  have 
both  stated  that  the  current  criterion  often  creates  work¬ 
load  management  problems  (4;  10). 

Literature  Review 

This  research  is  an  exploratory  effort  in  the  area 
of  criteria  for  pricing  staff  action.  While  no  literature 
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is  available  in  the  specific  topic  area,  some  related  data 
are  available  in  the  general  pricing  area.  A  survey  of 
pricing  personnel  conducted  in  FY  76  indicated  that  all 
major  procurement  directorates  within  the  Air  Force  use  a 
dollar  threshold  as  the  sole  criterion  for  pricing  staff 
action  (6:23).  The  study  further  indicated  that  in  some 
cases  pricing  staff  action  was  performed,  when  requested 
by  the  CO,  on  actions  below  the  threshold.  In  addition, 
the  study  revealed  that  in  times  of  excessive  workload, 
such  as  the  end  of  the  fiscal  year,  pricing  staff  action 
was  waived  on  efforts  above  the  threshold.  In  both  of 
these  instances,  criteria  other  than  threshold  were  con¬ 
sidered;  however,  the  survey  did  not  discuss  these  specific 
criteria.  As  a  result,  we  contacted  contracting  and 
pricing  personnel  in  an  attempt  to  identify  the  criteria 
used . 

For  actions  below  the  threshold,  criteria  cited 
most  often  were  type  of  contract,  type  of  effort,  and 
uniqueness  of  requirement  (12;  19).  While  these  criteria 
were  also  considered  important  in  the  decision  to  waive 
pricing  staff  action  on  efforts  above  the  threshold,  they 
were  considered  less  important  than  the  experience  of  the 
CO  and  the  degree  of  competition  (12;  13;  19).  Both 
contracting  and  pricing  personnel  considered  CO  experience 
to  be  the  most  critical  factor  in  the  decision  of  pricing 
to  waive  pricing  staff  action. 
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The  interviews  with  contracting  and  pricing 
personnel  showed  that  while  criteria  other  than  dollar 
threshold  are  used  in  unusual  cases,  there  are  no  set 
procedures  in  government  pricing  offices  to  regularly 
incorporate  these  criteria  into  the  management  process. 
However,  such  set  procedures  do  exist  within  the  defense 
industry. 

Major  defense  contractors,  for  the  most  part,  in 
buying  from  their  own  suppliers  have  contracting  and 
pricing  functions  similar  to  those  in  the  government. 

Our  interviews  with  industry  representatives  revealed 
that  while  dollar  threshold  is  used  as  one  criterion, 
it  is  not  usually  the  primary  or  driving  factor.  For 
instance,  at  the  Defense  Space  and  Systems  Group  of  TRW, 
Inc.,  the  primary  criterion  is  type  of  effort.  All  study 
proposals  and  all  space  component  efforts  require  pricing 
staff  action  regardless  of  dollar  value.  Further,  the 
type  of  contract  and  the  proposed  contractor  are  considered 
more  important  than  dollar  value  (3).  Similar  criteria 
are  used  by  other  major  defense  industries.  At  one  major 
electronics  manufacturer  a  dollar  threshold  is  used  as  the 
primary  criterion.  However,  a  number  of  different  thres¬ 
holds  are  used  based  upon  the  type  of  contract  and  the 
degree  of  competition. 
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The  data  obtained  from  the  pricing  management 
study  and  from  the  interviews  indicated  that  use  of 
other  criteria  in  addition  to  cost  was  operationally 
feasible  and  could  result  in  improvement  in  pric'  g 
management.  Therefore,  this  area  was  investigated. 

The  remaining  chapters  of  this  thesis  discuss  Che 
achievement  of  our  research  objectives.  Chapter  2--Metho- 
dology--d iscusses  the  methods  which  were  used  for  data 
collection  and  analysis.  Chapter  3--Data  Summary  and 
Analys is--presents  the  results  of  the  interviews  conducted 
and  the  results  of  the  statistical  tests  performed. 

Chapter  ^  —  Findings  and  Recommendat ions- -d i scusses  our 
proposed  model  based  upon  interview  results  and  provides 
suggestions  for  further  study. 
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Chapter  2 


METHODOLOGY 

This  chapter  describes  the  universe,  population  and 
sample  from  which  data  were  collected  for  this  research, 
the  techniques  employed  in  collecting  the  data,  the  statis¬ 
tical  tests  used  in  the  analysis  of  those  data,  the 
development  of  the  model,  and  research  assumptions  and 
1 imi tat  ions . 

Description  of  Universe.  Population  and  Sample 

The  universe  for  this  research  project  cons  i  s t s  of 
all  Air  Force  personnel  who  perform  pricing  actions.  This 
includes  not  only  the  "pricing  experts"  who  perform  pricing 
as  their  primary  activity,  but  also  contracting  officers 
and  buyers  who  perform  pricing  tasks  on  an  intermittent 
basis.  The  primary  population  of  interest  is  the  group  of 
"pricing  experts".  We  define  the  population  to  include  all 
personnel  above  journeyman  level  assigned  to  selected 
pricing  offices  within  AFLC  and  AFSC.  AFLC  has  five  pricing 
offices,  one  for  each  Air  Logistics  Center  (ALC) .  AFSC  has 
ten  pricing  offices,  three  of  which  are  located  at  major 
procurement  centers  (ASD.  ESD.  SAMSO)  as  defined  by  HQ  USAF 
(6:3).  The  primary  population  was  limited  to  those  pricing 
personnel  assigned  to  the  five  ALCs,  the  three  major  AFSC 
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offices,  and  two  non-major  offices  (see  Appendix  B) . 

Since  this  population  includes  those  offices  from  the  two 
acquisition  commands  which  are  responsible  for  the  majority 
of  Air  Force  contracting  and  includes  both  very  large  and 
very  small  offices,  generalization  of  results  of  this  study 
to  the  universe  is  considered  valid. 

A  sample  of  37  members  of  the  primary  population 
was  taken.  A  stratified  random  sample  was  used  to  assure 
that  members  from  each  of  the  offices  in  the  primary  popu¬ 
lation  were  selected  in  the  sample.  Current  personnel 
rosters  and  a  random  number  table  were  used  to  generate 
the  sample. 

The  prime  objective  in  sampling  was  to  select  a 
group  of  people  who  would  provide  responses  which  could  be 
considered  to  be  representative  of  the  population  (16:157). 
Since  the  selected  sample  included  approx ima t e 1 y  one-third 
of  the  total  population  and  included  members  from  all  of 
the  different  elements  in  the  population,  the  sample  was 
considered  representative. 

Data  Collection 

Data  collection  was  performed  in  two  phases.  The 
first  phase  involved  the  ident i f icat ion  and  definition  of 
specific  criteria  considered  important  for  the  determination 
of  pricing  staff  actions  and  the  development  of  decision 


rules  to  be  used  to  apply  these  criteria.  The  second  phase 
involved  the  abstraction  of  the  criteria,  identified  in 
phase  I.  from  randomly  selected  contracts. 

Phase  I .  This  first  step  was  the  identification  and  defi¬ 
nition  of  criteria  which  pricing  experts  felt  should  be 
included  in  the  pricing  action  decision.  This  information 
was  obtained  through  a  survey  of  members  of  the  pricing 
sample.  The  survey  was  conducted  by  personal  and  telephone 
interviews.  While  a  review  of  existing  literature  is  an 
obvious  starting  point  in  the  research  of  any  problem,  in 
many  cases  only  a  small  portion  of  current  knowledge  of  the 
problem  has  been  put  into  writing.  Therefore,  much  can 
often  be  obtained  through  gathering  current  information 
from  people  knowledgeable  in  the  area.  One  method  of 
obtaining  such  information  is  to  conduct  a  survey  and  to 
solicit  ideas  from  experts  in  the  field  (8:84-85).  The 
survey  technique  was  selected  as  the  method  of  data  collec¬ 
tion. 

A  survey  may  be  conducted  on  a  one-to-one  basis, 
as  in  an  interview,  or  on  an  impersonal  basis  through  the 
use  of  a  ’questionnaire  (8  199).  The  interview  technique 
was  selected  over  a  questionnaire  for  a  number  of  reasons. 
First,  the  primary  questions  of  interest  were  open  ended 
and  very  subjective.  The  interviewees  were  asked  to  make 
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judgements  and  give  opinions.  Second,  the  initial  response 
to  a  question  often  determined  subsequent  questions.  And 
third,  many  of  the  terms  and  concepts  which  were  discussed 
were  quite  vague  and  ambiguous  and  therefore  needed  to  be 
explored  in  detail  with  each  interviewee  in  order  to 
develop  operational  definitions. 

Both  personal  and  telephone  interviews  were  con¬ 
ducted.  For  in-depth,  subjective  questions  requiring 
probing,  a  personal  interview  is  the  preferred  technique 
(9:209).  However,  due  to  the  wide  geographic  dispersion 
of  the  sample  members,  personal  interviews  with  all  members 
in  the  sample  were  not  possible  within  the  limitations  of 
time  and  funds.  Therefore,  personal  interviews  were  con¬ 
ducted  with  people  located  at  Wr  ight-Patterson  AFB.  and 
telephone  interviews  were  conducted  with  those  people 
located  elsewhere. 

The  use  of  both  telephone  and  personal  interviews 
presupposes  that  replies  will  be  the  same  irrespective  of 
the  interview  method.  While  this  is  true  in  most  cases, 
some  studies  have  suggested  that  on  questions  of  attitudes 
the  presence  of  a  personal  interviewer  "...  may  inter¬ 
ject  a  modifying  element  in  the  interview  situation 
[9:212]."  However,  for  this  research,  the  "interjection 
of  a  modifying  element"  is  not  considered  a  serious  problem 
for  two  reasons.  First,  the  population  consists  of  highly 


trained  professionals  who  provided  professional  rather  than 
personal  opinions.  Second,  the  interviewers  followed  the 
same  interview  guide  for  both  the  personal  and  telephone 
int  er  v  i  ews  . 

The  interview  guide  (see  Appendices  C  and  D)  con¬ 
sisted  of  a  cover  letter  containing  an  introduction  and 
explanation  of  the  purpose  of  the  research  and  a  list  of 
questions.  The  questions  were  used  to  obtain  general  back¬ 
ground  information  and  to  establish  a  baseline  for  further 
questioning.  These  questions  covered  such  items  as  the 
value  of  current  thresholds,  the  basis  for  the  thresholds, 
and  whether  or  not  contracts  under  the  threshold  were 
reviewed  and  if  contracts  over  the  threshold  were  returned 
to  the  CO  for  analysis. 

Additional  questions  were  asked  based  upon  responses 
to  those  questions  listed  in  the  interview  guide  and  were 
directed  toward  obtaining  criteria  and  recommendations  for 
improved  criteria.  Questions  centered  on  identifying, 
defining,  and  ranking  recommended  criteria.  Whenever  a 
large  number  of  criteria  were  identified  during  the  inter¬ 
views.  the  interviewees  were  instructed  to  rank  only  the 
five  which  they  considered  to  be  most  significant.  This 
was  done  since  studies  have  indicated  that  ranking  more 
than  five  factors  is  usually  difficult  and  results  in 
unreliable  rankings  (8:241). 
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Obtaining  definitions  of  recommended  criteria  was 
a  major  part  of  the  interviews.  Based  upon  previously  con¬ 
ducted  interviews,  it  appeared  that  many  of  the  recommended 
criteria  used  terms  which  did  not  have  a  commonly  accepted 
and  understood  definition.  For  instance  the  terms  "com¬ 
plexity”  and  "CO  experience"  had  both  been  previously 
suggested  as  possible  criteria.  Since  these  terms  would 
require  operational  definitions  if  they  were  to  be  used  in 
any  model,  the  interviews  were  used  to  determine  what  fac¬ 
tors  make  an  effort  "complex"  or  a  buyer  "experienced." 

In  addition  to  obtaining  identification  and  defi¬ 
nition  of  criteria,  a  decision  rule  for  each  selected 
criterion  was  also  obtained.  The  decision  rule  was  a 
statement  of  how  the  selected  criteria  would  be  applied 
in  making  a  pricing  action  decision. 

The  interview  guide  was  tested  prior  to  its  use. 

A  test  ensures  that 

.  .  .  the  questions  meet  the  objectives  of  the 
survey;  all  important  phases  of  the  survey  have 
been  adequately  covered;  the  questions  stimulate 
respondent  cooperation;  the  questions  are  in  satis¬ 
factory  order;  and  the  questions  are  completely 
understood  by  the  respondents  [21:2-31. 

The  guide  was  tested  on  Air  Force  Institute  of 
Technology  (AFIT)  procurement  instructors  from  both  the 
graduate  and  the  continuing  education  programs.  Personal 
interviews  were  conducted  with  the  instructors  followed 


19 


by  a  discussion  of  the  interview  itself.  While  the  AFIT 
instructors  were  not  part  of  the  population  of  pricing 
experts,  they  are  familiar  with  both  pricing  policies  and 
procedures  and  proper  research  methods  and  therefore  we 
considered  the  instructors  to  be  a  valid  test  group. 

Changes  recommended  by  the  test  group  were  considered  for 
inclusion  in  the  interview  guide.  A  field  test  of  the 
interview  guide,  with  members  of  the  population,  was  not 
considered  feasible  due  to  the  limited  size  of  the  popu¬ 
lation. 

Support  and  cooperation  from  the  interviewees  were 
essential  to  this  research,  since  the  interviews  required 
considerable  thought  and  time  from  the  respondents.  In 
order  to  assure  this  support,  two  steps  were  taken.  First, 
HQ  USAF  notified  the  major  commands  which  in  turn  notified 
the  director  of  each  office  and  requested  his  support  prior 
to  the  interviews.  Second,  the  interviewers  provided 
advance  notice  to  all  interviewees  and  established  a  time 
convenient  to  the  interviewee  for  the  interview. 

The  data  obtained  from  the  interviews  were  analyzed 
in  order  to  determine  the  criteria  which  are  considered 
important  in  the  decision  to  provide  pricing  staff  action. 
These  selected  criteria  and  the  decision  rules  were  used 
in  Phase  II  of  data  collection  and  were  used  to  develop  a 
decision  model  in  Phase  II  of  data  analysis. 
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Phase  II.  Phase  II  involved  the  collection  of  contract 


data  for  use  in  applying  the  decision  rules  obtained  from 
Phase  I.  Information  concerning  the  selected  criteria  and 
decision  rules  was  abstracted  from  contract  files. 

A  stratified  random  sample  of  50  contracts  awarded 
in  FY  78  was  made.  The  sample  included  contracts  from  one 
major  AFSC  office,  one  non-major  AFSC  office,  and  one  AFLC 
office.  Half  of  the  contracts  selected  were  selected  from 
those  which  were  above  the  threshold  and  analyzed  by  the 
pricing  staff  and  half  from  those  below  the  threshold  which 
were  not  analyzed  by  the  pricing  staff.  Listings  of  com¬ 
pleted  contracts  from  each  office  and  a  random  number  table 
were  used  to  select  the  contracts  based  upon  the  last  four 
digits  of  the  Procurement  Instrument  Identification  Number. 
This  stratified  sampling  plan  was  selected  to  assure  that 
a  broad  range  of  dollar  values  and  types  of  buys  were 
included  in  the  sample. 

We  abstracted,  from  each  contract,  information 
pertaining  to  the  criteria  identified  by  the  price  analysts 
and  ranked  highest  by  the  statistical  tests.  This  infor¬ 
mation  was  subsequently  used  in  Phase  II  of  the  data 
analys is . 
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Data  Analysis 


Data  analysis  was  performed  in  two  phases,  with 
each  phase  corresponding  to  a  data  collection  phase. 

Phase  I.  The  data  obtained  from  the  first  phase  of  inter¬ 
views  were  analyzed  to  identify  and  rank  the  decision 
criteria.  This  phase  of  analysis  determined  the  existence 
and  degree  of  association  or  agreement  between  the  ranked 
criteria  provided  by  the  interviewees.  The  degree  of  asso¬ 
ciation  was  examined  in  two  groups. 

Group  I--within  the  population  of  pricing  experts 

Croup  II--between  subsets  of  population  AFLC  or 
AFSC  . 

This  test,  a  measure  of  consistency  within  the 
pricing  community,  was  necessary  in  order  to  generalize 
the  results  of  the  pricing  population.  The  second  test 
was  performed  to  determine  if  the  differences  in  buying 
mission  between  AFSC  and  AFLC  affected  decision  criteria. 
Both  tests  were  performed  using  Kendall  Coefficient  of 
Concordance.  (See  Appendix  H.) 

The  Kendall's  Coefficient  of  Concordance  is  a 
measure  of  the  agreement  of  rankings  of  a  number  of  sub¬ 
jects  and  therefore  in  this  study  we  used  the  Kendall  test 
to  determine  the  degree  of  agreement  among  the  criteria 
provided  by  interviewers.  The  coefficient  equals  one  when 
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there  is  perfect  agreement  and  zero  when  there  is  no  agree¬ 
ment  (14:148).  The  hypothesis  tested  was: 

Hq.-  No  agreement  exists 

Ha:  Agreement  does  exist. 

Through  the  use  of  this  test,  a  significance  level 
was  determined.  A  small  significance  level  indicates  that 
consistency  or  agreement  exists  and  that  the  null  hypothesis 
may  be  rejected  (20:26).  The  results  of  the  test  permitted 
the  rejection  of  the  null  hypothesis  which  indicated  that 
the  rankings  were  a  valid  basis  for  further  study.  Detailed 
analysis  is  contained  in  Chapter  3. 

The  data  from  Phase  I  were  also  analyzed  to  provide 
a  decision  model.  We  had  intended  to  develop  the  model 
based  upon  decision  tree  analysis,  using  the  criteria  and 
decision  rules  obtained  in  Phase  I.  The  model  was  to  be 
limited  to  a  maximum  of  five  factors  and  include  only  those 
criteria  which  had  been  provided  decision  rules  by  at  least 
51  percent  of  the  interviewees  in  response  to  question  ten 
of  the  interview  guide.  Each  criterion  was  to  be  estab¬ 
lished  as  a  decision  point  in  the  decision  tree,  and  at 
each  point  a  decision  would  be  made  as  to  whether  the 
proposal  was  to  be  sent  to  the  pricing  staff  or  retained 
in  the  buying  office.  However,  due  to  the  nature  of  the 
decision  rules  provided  by  the  interviewees,  we  were  unable 
to  employ  the  decision  tree  approach.  The  rules  obtained 


from  Che  pricers,  in  almost  every  case,  involved  a  caveat, 
which  prevented  its  use  in  decision  tree  analysis.  As  a 
result,  we  developed  a  model  consisting  of  the  five  selected 
factors,  but  which  called  for  a  "majority  rule"  decision 
rather  chan  a  decision  tree  approach. 

Phase  1 1 .  Data  collected  in  the  second  phase  were  analyzed 
to  determine  whether  or  not  the  use  of  the  decision  model, 
based  upon  the  selected  decision  r u les , r esu 1  ted  in  decisions 
different  from  those  made  using  the  dollar  threshold.  The 
determination  of  differences  was  accomplished  through  the 
comparison  of  actual  decisions  made  using  the  dollar  thres¬ 
hold  to  decisions  we  arrived  3t  using  the  model.  Results 
discussed  in  Chapter  3  show  that  the  use  of  the  model  did 
result  in  different  decisons  than  those  obtained  under  the 
thr esho  Id . 

While  our  analysis  indicated  that  the  model  pro¬ 
duced  different  results,  it  could  not  indicate  if  the 
results  were  "better".  However,  since  the  model  identifies 
contracts  which,  in  the  opinion  of  the  experts,  received 
action  by  the  wrong  group  (pricing  staff  when  COs  should 
have  handled  them  or  visa  versa)  then  we  assumed  that  the 
model  provided  for  a  better  utilization  of  pricing 
resources . 
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Assumpt ion s/Limi tat  ions 


Assumptions 

1.  The  samples  of  pricing  personnel  were  repre¬ 
sentative  of  the  Air  Force  contracting  community  and 
therefore  results  can  be  generalized. 

2.  Selected  statistical  tests  were  appropriate 
for  the  type  and  nature  of  the  data  obtained. 

3.  Interviewees  answered  questions  honestly  and 
conscientiously  and  have  the  knowledge  necessary  to  pro¬ 
vide  valid  opinions  and  decisions. 

Limi tat  ions 

1.  Time  and  resources  available  to  researchers. 

2.  Time  provided  by  interviewees. 

3.  Access  to  offices  located  outside  of  Wright- 
Patterson  AFB.  OH. 
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Chapter  3 


DATA  SUMMARY  AND  ANALYSIS 

This  chapter  discusses  the  data  collected  and  the 
analysis  of  that  data.  This  chapter  contains  a  discussion 
of  interview  questions,  a  statistical  and  descriptive  sum¬ 
mary  of  interview  results,  and  an  analysis  of  the  statis¬ 
tical  tests  performed  on  interview  results.  Our  findings 
and  recommendations  based  upon  these  analyses  are  contained 
in  Chapter  4 . 

Interview  Procedures 

Interviews  were  conducted  with  the  pricing  personnel 
in  accordance  with  our  research  design  outlined  in  Chapter 
2.  Thirty  interviews  were  conducted  out  of  the  selected 
sample  of  37  personnel.  One  person  refused  to  participate 
in  an  interview  on  the  basis  that  he  felt  that  research 
studies  in  the  Air  Force  were  too  frequent  and  did  not  pro¬ 
duce  any  results.  Three  people  had  left  Government  service 
or  had  been  transferred  out  of  the  pricing  office,  and  three 
were  on  leave  or  travel  status  during  the  interview  period. 
The  remaining  30  represent  a  stratified  sample  of  the  popu¬ 
lation  and  are  considered  representative. 


We  found  that  the  pricing  personnel  who  partici¬ 
pated  in  this  research  effort  were  very  helpful  and  co¬ 
operative.  Many  had  prepared  written  responses  to  the 
questions  on  the  interview  guide  prior  to  the  interview. 
Further,  although  the  questions  asked  for  only  opinions 
and  best  estimates,  many  of  the  pricers  checked  historical 
files  and  records  prior  to  responding  to  the  questions.  On 
the  average,  the  interviews  lasted  approximately  45  minutes, 
yet  all  of  the  interviewees  were  willing  to  spend  the  time 
needed  to  answer  the  questions. 

General  Questions 

The  interviews  consisted  of  ten  basic  questions  as 
outlined  in  the  interview  guide  (Appendix  D) .  These  ques¬ 
tions  were  used  as  a  starting  point  for  further  questions. 
Additional  questions  and  issues  were  raised  to  determine 
the  rationale  and  reasoning  behind  the  responses  to  the 
first  ten  questions.  The  questions  were  asked  in  the 
sequence  shown  on  the  guide  since  latter  questions  built  on 
earlier  ones  We  tabulated  the  responses  to  each  question 
both  by  command  and  by  total  Air  Force  response.  We  ana¬ 
lyzed  the  responses  to  determine  trends  and  also  to  deter¬ 
mine  if  the  research  objectives  of  each  question  had  been 
met . 


Quest  ion  *  1 .  We  began  each  interview  with  a  general  dis¬ 
cussion  of  the  pricing  dollar  threshold.  Our  objective  of 
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the  first  question  wjs  to  focus  the  interviewee's  thinking 
on  the  reason  for  and  basis  of  the  threshold.  The  first 
question  was  in  two  parts.  The  first  part  sought  to  deter¬ 
mine  the  rationale  for  the  use  of  any  threshold,  in  lieu 
of  the  use  of  possible  other  criteria,  as  the  primary  deci¬ 
sion  rule.  We  were  only  somewhat  successful  in  meeting 
this  objective  with  part  one  of  the  question.  Most  of  the 
pricers’  initial  responses  dealt  with  the  rationale  for  the 
specific  dollar  value  of  the  threshold,  rather  than  with 
the  rationale  for  the  threshold  itself.  Even  after  we 
clarified  the  question,  some  responses  still  dealt  with 
the  specific  dollar  amount.  This  is  shown  below  in  the 
table  of  responses. 

QUESTION  la  IN  YOUR  OPINION.  WHAT  IS  THE  BASIS  FOR  THE 


USE  OF  A  DOLLAR  THRESHOLD  AS  THE  DECISION 

CRITERIA? 

RESPONSE 

AFLC 

AFSC 

COMBINED 

Easy  to  apply 

36" 

50* 

44* 

All  rules  by  threshold 

7 

11 

9 

Relates  to  complexity 

29 

17 

22 

Required  by  ASPR 

64 

6 

32 

Focuses  on  potential  savings 

21 

39 

31 

NOTE  Totals  for  each  command 

because  multiple  responses  were 

add  to  more 
a  1 lowed . 

than  100 

percent 
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Of  the  categories  above,  we  considered  the  first 
three  to  deal  with  the  rationale  for  the  threshold  itself 
and  the  last  two  to  deal  with  the  rationale  for  the  specific 
dollar  amount  of  the  threshold.  Of  those  responses  dealing 
with  the  threshold  itself,  the  majority  focused  on  the  ease 
of  application  as  the  basis  for  the  use  of  a  threshold.  A 
dollar  threshold  is  easy  and  convenient  to  use.  Since  this 
was  regularly  cited  as  a  major  benefit  of  the  use  of  a 
threshold,  the  ease  of  application  of  any  alternative  is  an 
important  consideration. 

The  pricers  also  indicated  that  the  relationship 

■\ 

of  dollar  value  to  complexity  is  an  important  basis  for  the 
threshold.  We  believe  that  this  indicates  that  complexity 
should  be  considered  in  the  pricing  decision  and  that  dol¬ 
lar  value  is  an  effective  way  to  measure  complexity. 

However,  this  was  not  borne  out  by  later  questions.  A 
small  percentage  of  the  responses  focused  on  the  perva¬ 
siveness  of  dollar  thresholds  throughout  the  contracting 
process.  Dollar  thresholds  are  used  as  a  basis  for  the 
inclusion  of  special  clauses,  for  the  requirement  for 
various  reviews,  and  for  the  obtaining  of  administrative 
approvals.  This  attitude  was  expressed  by  one  pricer  as. 

"We  use  dollar  thresholds  for  everything  else,  why  not 
pricing." 
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As  mentioned  above,  two  categories  of  responses 
dealt  with  the  rationale  for  the  specific  dollar  amount 
rather  than  with  the  rationale  for  the  use  of  a  dollar 
threshold.  We  believe  that  this  perhaps  indicates  that 
the  use  of  a  dollar  threshold  is  perceived  as  a  "given", 
or  in  other  words,  as  something  which  is  not  even  subject 
to  change.  One  of  the  responses  provided  indicates  that 
there  is  a  widespread  misconception,  particularly  among 
AFLC  pricers,  as  to  what  the  DAR  requires.  A  significant 
proportion  of  AFLC  pricers  (64  percent)  stated  that  the 
reason  for  the  threshold  is  that  the  DAR  established  it. 
This  is  plainly  incorrect.  The  DAR  does  set  $100,000.00 
as  the  level  at  which  some  form  of  price  analysis  must  be 
performed;  however,  the  DAR  does  not  state  by  whom  the 
pricing  must  be  done.  It  does  not  set  dollar  value  as  the 
sole  basis  for  making  that  decision. 

In  AFLC.  HQ  AFLC  Pricing  Policy  has  established 
a  $100,000.00  threshold  for  all  AFLC  pricing  offices.  We 
feel  that  this  contributes  to  the  misconception  since  the 
offices  see  the  threshold  as  directive  and  not  subject  to 
adjustment.  This  misconception  is  also  evident  in  the 
responses  to  part  two  of  the  question. 

The  objective  of  part  two  of  the  question  was  to 
determine  the  rationale  for  the  specific  dollar  amount 


used  in  each  pricing  office.  While  the  dollar  amount  is 
$100,000.00  in  all  of  the  AFLC  offices,  the  amounts  vary 
significantly  within  AFSC.  The  dollar  thresholds  for  the 
AFSC  offices  are  shown  below 


SAMSO . $  500.000 

ESD .  200,000  (R6.D) 

.  500,000  (Systems) 

ASD .  2,000,000 

ASD/R&D  .  500,000 

ADTC .  500,000 


Prior  to  conducting  the  interviews,  we  believed 
that  the  dollar  level  was  related  to  workload  and  to  type 
of  item  purchased,  since  this  seemed  to  explain  the  dif¬ 
ference  between  the  commands.  The  results  of  question  lb 
indicate  that  this  is  partially  true. 

QUESTION  lb:  IN  YOUR  OPINION.  WHAT  IS  THE  BASIS  FOR  THE 
SELECTION  OF  THE  PARTICULAR  DOLLAR  VALUE  USED  AS  THE 
THRESHOLD? 


RESPONSE 

AFLC 

AFSC 

COMBINED 

Due  to  workload 

141 

721 

471 

Related  to  review  levels 

14 

22 

19 

Set  by  r egu lat ion/ law 

72 

17 

41 

Related  to  inflation 

0 

11 

6 

The  results  show  that  the  basis  for  the  specific 
dollar  amount  of  the  threshold  varies  significantly  by  com¬ 
mand.  This  may  reflect  the  type  of  item  being  purchased 
since  the  commands  purchase  very  different  type  of  products. 
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As  mentioned  earlier,  HQ  AFLC  Pricing  Policy  has 


set  a  $100,000.00  threshold  for  all  ALC  offices,  and  there¬ 
fore  the  individual  pricing  offices  do  not  have  the 
authority  to  adjust  the  threshold.  This  is  reflected  in 
the  low  AFLC  response  of  inflation  and  workload  as  deter¬ 
minants  of  the  dollar  amount  for  the  threshold.  Again,  the 
misconception  that  the  level  has  been  established  by  the 
DAR  is  prevalent,  as  indicated  by  the  72  percent  response 
to  this  answer . 

In  AFSC,  on  the  other  hand,  each  office  sets  its 
own  threshold  level.  While  the  majority  of  the  offices 
originally  established  the  level  at  $100,000.00,  the  offices 
have  increased  the  threshold  over  time.  The  increase  has 
been  due  mainly  to  workload,  with  a  small  percentage  of  the 
increase  due  to  the  effects  of  inflation. 

In  both  commands,  a  small  number  of  the  pricers 
cited  other  review  levels  as  the  basis  for  the  dollar 
amount  of  the  pricing  threshold.  We  believe  that  the  dol¬ 
lar  levels  are  set  the  same  for  the  number  of  staff  actions, 
such  as  reviews  and  pricing,  in  order  to  avoid  confusion 
and  for  admin i s tr a t i ve  ease. 

Question  >2.  With  the  first  question,  we  established  the 
basis  for  the  use  of  the  dollar  threshold  decision  rule. 

In  our  second  and  third  questions  of  the  interviews,  we 


attempted  to  determine  if  the  rule  was  always  followed, 
and  if  not.  under  what  circumstances  was  it  not  followed. 
Prior  to  the  interviews,  we  believed  that  the  threshold 
rule  was  most  likely  not  always  strictly  adhered  to, 
mainly  due  to  workload  problems.  Further,  we  felt  that 
by  examining  the  instances  when  the  rule  was  not  followed, 
we  could  gain  insight  into  the  decision  making  processes 
used  in  pricing.  We  used  question  *2  as  a  lead-off  ques¬ 
tion  for  this  topic. 

QUESTION  #2:  DO  YOU  KNOW  OF  ANY  PRICING  ACTION  ACCEPTED 
3Y  THIS  OFFICE  WHICH  DID  NOT  FOLLOW  THE  DOLLAR  THRESHOLD 
RULE?  IN  OTHER  WORDS.  CONTRACTS  ABOVE  THE  THRESHOLD 
BEING  RETURNED  TO  THE  CONTRACTING  OFFICER.  OR  CONTRACTS 
BELOW  THE  THRESHOLD  BEING  ACCEPTED  BY  THIS  OFFICE? 

As  we  expected,  we  found  that  the  rule  was  not 
followed  in  all  cases. 


RESPONSE 

AFLC 

AFSC 

COMBINED 

Accept 

cases  below  threshold 

93X 

78* 

84X 

Reject 

hold 

cases  above  the  thres- 

64X 

UU  X 

Reject 
a  pa 

above,  but  only  with 
rt  ial 

0 

} 

39X 

837. 

81X 

The  responses  indicate 

that  a 

lmost  all 

of 

the 

offices  do  accept  cases  below  the  threshold.  However, 
this  does  not  mean  that  the  pricers  are  handling  below  the 
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threshold  cases  as  normal  price  actions.  Rather,  on  these 
cases,  the  pricers  provide  advice  and  guidance,  assist  in 
the  use  of  special  techniques  such  as  the  learning  curve, 
and  may  review  the  final  Price  Negotiation  Memorandum  (PNM). 
However,  the  final  responsibility  for  the  case  still  rests 
with  the  contracting  officer  who  must  prepare  the  negoti¬ 
ation  objective,  handle  negotiations,  and  write  the  PNM. 

As  one  pricer  stated,  "we'll  hold  the  CO's  hand  and  provide 
moral  support,  but  we  won't  do  all  the  work  for  him." 

At  least  some  of  the  personnel  in  every  pricing 
office  stated  that  actions  above  the  threshold  were 
returned  to  the  contracting  officer  because  the  workload 
in  the  pricing  office  was  too  great.  In  most  instances, 
these  cases  were  returned  without  any  action  at  all.  The 
most  that  was  offered  was  assistance  as  would  normally  be 
provided  on  a  below  the  threshold  case.  However,  in  one 
pricing  office  (SAMSO) .  no  cases  were  returned  without 
some  form  of  pricing  analysis.  Every  file  returned 
received  at  least  a  partial  review  consisting  of  a  summary 
of  audit  findings,  a  listing  of  potential  problem  areas, 
and  an  indication  of  possible  negotiation  approaches.  We 
believe  that  this  procedure  represents  an  excellent  com¬ 
promise  between  the  use  of  limited  resources  and  the  need 
to  provide  assistance  to  the  contracting  officers. 


An  interesting  phenomenon  was  noted  in  two  separate 


offices.  In  both  offices  the  chief  of  the  office  stated 
that  no  cases  were  returned  without  action  that  were  above 
the  dollar  threshold.  Yet  pricers  in  these  offices  indi¬ 
cated  that  they  had  personally  returned  a  number  of  such 
cases.  This  indicates  both  a  lack  of  standard  procedures 
within  the  office  and  a  lack  of  communication. 

Quest  ion  #3 .  With  the  second  question  we  determined  that 
the  dollar  threshold  was  not  always  followed.  We  then 
attempted  to  determine  when  and  why  it  was  not  followed. 
Question  3  examined  these  issues.  The  first  part  of  this 
question  was  aimed  at  determining  the  percentage  of  pricing 
actions  which  did  not  follow  the  rule.  Our  purpose  was  to 
determine  if  non-adherence  to  the  dollar  th.eshold  rule 
was  a  common  occurrence.  The  results  indicate  that  non¬ 
adherence  is  fairly  common. 

QUESTION  3a  WHAT  PERCENTAGE  OF  THE  TOTAL  NUMBER  OF  CON¬ 
TRACTS  ACCEPTED  BY  THIS  OFFICE  DOES  NOT  FOLLOW  THE  THRES¬ 
HOLD  RULE? 


RESPONSE 

AFLC 

AFSC 

COMBINED 

Less  than  ten  percent  of 

cases 

H 

sO 

ao 

391 

59* 

Between  ten  and  thirty 

percent 

7 

17 

13 

Above  thirty  percent 

7 

uu 

28 

The  responses  seem  to  indicate  that  AFSC  returns 
a  much  higher  percentage  of  cases  than  does  AFLC.  We 
believe  that  this  indicated  that  AFSC  had  a  much  more 
severe  workload  problem  than  does  AFLC.  This  belief  was 
later  supported  by  responses  to  question  3c. 

The  responses  to  this  question  (3a)  varied  greatly 
even  within  the  same  office.  We  believe  this  is  due  to 
the  fact  that  an  individual  pricer  would  only  have  know¬ 
ledge  of  the  number  of  contracts  he  personally  handled 
which  did  not  follow  the  rule,  and  would  not  necessarily 
be  aware  of  the  percentage  of  an  office  wide  basis. 

After  determining  that  the  dollar  threshold  rule 
was  not  followed  in  a  relatively  large  number  of  cases,  we 
attempted  to  determine  who  made  the  decision  to  accept  a 
case  below  the  threshold  or  return  a  case  above  the  thres¬ 
hold.  Our  primary  purpose  with  this  question  was  to  deter¬ 
mine  if  the  contracting  personnel  were  involved  in  this 
dec i s i on . 

QUESTION  3b:  WHO  MAKES  THE  DECISION  TO  ACCEPT/ RETURN  THESE 
ACTIONS? 


RESPONSE 

AFLC 

AFSC 

COMBINED 

Pricing  Chief 

86Z 

17Z 

4  71 

Price  Analyst 

7 

38 

25 

Pricing  Croup 

7 

28 

19 

Joint  decision  with  contracting 

0 

17 

9 

36 


The  results  indicate  that  the  pricing  office  alone, 
without  contracting  officer  input,  makes  the  decision  in 
almost  all  cases  (91  percent).  It  appears  that  in  AFSC 
the  price  analysts  are  delegated  somewhat  more  authority 
to  make  these  decisions  than  in  AFLC  where  most  of  the 
responsibility  rests  with  the  pricing  chief.  In  one  AFSC 
office,  a  panel  of  pricing  analysts  and  branch  chiefs 
review  all  contracts  being  considered  for  return  to  the 
contracting  officer,  and  a  joint  decision  of  all  the 
pricing  personnel  is  made.  We  believe  that  this  allows 
for  more  efficient  use  of  pricing  personnel  since  it  allows 
for  the  shifting  of  workload  among  the  individual  pricers. 

In  only  one  office  is  contracting  involved  directly  in  the 
initial  decision  making  process.  However,  in  discussions 
following  this  question,  we  found  that  the  contracting 
officer  has  the  final  decision  in  almost  all  cases.  A 
pricing  decision  to  return  a  case  above  the  threshold  is 
actually  only  a  recommendation  since  the  contracting  officer 
can  refuse  to  accept  the  case  back. 

Since  the  interviews  up  to  this  point  clearly  indi¬ 
cated  that  the  pricers  were  making  decisions  using  some 
criteria  other  than  just  dollar  threshold,  our  next  question 
attempted  to  determine  what  factors  were  used  in  making  that 
decision.  The  purpose  of  the  question  was  only  to  determine 
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what  factors  were  considered  important  by  the  pricers.  In 
this  question  we  did  not  attempt  to  obtain  definition  of 
the  factors  or  a  ranking  or  prioritization  of  the  factors. 
We  believed  that  these  items  were  not  necessary  at  this 
point  in  the  interviews  since  the  factors  mentioned  in 
response  to  this  question  would  be  used  only  as  a  founda¬ 
tion  for  later  questioning. 

QUESTION  3c:  ON  WHAT  BASIS  IS  THE  DECISION  MADE? 


RESPONSE 

AFCL 

AFSC 

COMBINED 

Workload 

211 

721 

50* 

Complex i ty 

71 

22 

44 

Contracting  officer 
exper ience 

29 

56 

44 

Contractor 

0 

6 

3 

Dollar  value 

21 

11 

16 

Completeness  of 
proposal 

36 

39 

38 

Political  pressure 

7 

17 

13 

Type  of  contract 

29 

22 

19 

Amount  of  competition 

14 

11 

12 

Performance  period 

7 

6 

6 

Special  provisions 

14 

22 

19 

NOTE:  Percentages  add  to  more  than  100  percent  because 

multiple  responses  were  allowed. 


As  was  expected,  workload  was  listed  as  the  most 
important  factor.  This  is  logical  si^ce  no  decision  would 
have  to  be  made  if  workload  was  not  a  problem.  The  response 
to  this  question  seems  to  indicate  that  AFLC  has  a  much 
lower  workload  problem  than  does  AFSC.  This  was  also 
indicated  in  question  3a  which  showed  that  the  majority  of 
the  pricers  in  AFLC  felt  that  less  than  ten  percent  of  the 
contracts  were  returned  to  the  contracting  officers,  while 
the  majority  of  the  pricers  in  AFSC  felt  that  more  than  30 
percent  of  the  contracts  were  returned. 

While  AFSC  cited  workload  as  the  primary  factor, 

AFLC  cited  complexity  most  often.  In  all  of  the  other 
factors,  the  commands  showed  little  difference.  Detailed 
discussions  of  these  factors  and  the  basis  for  the  factors 
are  provided  in  the  analysis  of  question  *7,  *8,  and  *9. 

Our  final  question  in  the  area  of  non-adherence  to 
the  threshold  rule  was  to  determine  if  the  individual 
pricing  offices  had  a  written  policy  of  how  these  decisions 
should  be  made  We  found  that  only  one  office  in  both  AFSC 
and  AFLC  had  a  written  policy.  The  policy  was  a  pricing 
office  operating  instruction  which  was  prepared  by  the 
Chief  of  Pricing  for  internal  use  within  the  pricing  office. 


QUESTION  3d:  IS  THERE  A  WRITTEN  POLICY? 


RESPONSE 

AFLC 

AFSC 

COMBINED 

Written  policy 

297. 

07. 

13% 

No  written  policy 

71 

100 

87 

Question  **.  The  first  three  questions  indicated  that  con¬ 
tracting  officers  were,  on  a  somewhat  regular  basis,  having 
to  analyze  pricing  cases  which  were  above  the  threshold.  In 
other  words,  contracting  officers  were  performing  price 
analysis  which  according  to  office  instructions  should  have 
been  handled  by  the  pricers.  Since  this  was  the  case,  we 
sought  to  determine  if  there  was  any  perceived  difference 
between  the  quality  of  the  work  performed  by  the  price  ana¬ 
lysts  as  compared  to  the  work  performed  by  the  contracting 
officers.  Our  purpose  here  was  to  find  out  if  the  pricers 
had  any  special  skills  or  unique  talents  above  those  of  the 
contracting  officers. 

QUESTION  #4:  DO  YOU  FEEL  THAT  THERE  IS  A  DIFFERENCE  IN  THE 
EFFECTIVENESS  (ACCURACY.  COMPLETENESS,  TIMELINESS.  DOLLAR 
SAVINGS  TO  THE  GOVERNMENT.  ETC.)  OF  PRICING  ACTIONS  PERFORMED 
BY  THIS  OFFICE  AS  COMPARED  TO  FRIC1NG  ACTIONS  PERFORMED  BY 
THE  CONTRACTING  OFFICERS? 

Since  we  asked  this  question  only  of  the  pricers  we 
expected  an  overwhelming  "yes”  response,  and  we  received 
one.  Every  pricer  stated  that  pricers  performed  a  better 
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job  than  contracting  officers  in  the  area  of  pricing.  Most 
felt  that  pricers  did  significantly  better.  Typical  com¬ 
ments  included  a  comparison  of  pricers  to  buyers  as  analogous 
as  comparing  doctors  to  nurse*3.  Other  comments  included 
statements  such  as,  "CO ' s  just  don't  have  the  ability  to  do 
pricing,"  "Most  CO’s  don’t  like  pricing  and  therefore  do  a 
poor  job,"  and  "Many  times  the  CO ' s  just  give  away  the  farm." 

We  recognized  the  inherent  weakness  of  asking  the 
question  only  of  pricers.  We  expected  the  responses  to  be 
heavily  biased;  however,  we  included  the  question  mainly  as 
a  starting  point  to  determine  why  pricers  considered  them¬ 
selves  so  much  better  in  pricing  than  contracting  officers. 
Our  objective  was  to  determine  what  factors  made  the  pricers 
such  experts  in  the  field. 

QUESTION  4a:  WHY  ARE  PRICERS  BETTER  THAN  CO'S  IN  PERFORMING 
PRICING  ACTIONS? 


RESPONSE 

AFLC 

AFSC 

COMBINED 

Educational  background 

571 

49* 

53* 

Technical  Training 

43 

44 

44 

Exper i ence 

50 

44 

47 

Focus/attitude 

64 

56 

59 

NOTE:  Percentages  add  to  over  100  percent  since  multiple 

responses  were  allowed. 
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As  shown  dbove.  the  most  cited  dnswer  was  that  the 
"focus"  was  different  between  pricers  and  buyers.  By  this 
the  pricers  meant  that  the  two  groups  looked  at  the  task  of 
pricing  differently.  The  pricers  see  pricing  as  their 
primary  task  and  one  that  had  to  be  done  thoroughly  and 
completely  in  all  cases.  The  contracting  officers,  on  the 
other  hand,  see  pricing  as  just  one  of  many  tasks  which  must 
be  completed  in  order  to  get  the  contract  awarded.  It  has 
no  more  importance  than  any  other  contracting  task.  Further, 
pricers  stated  that  the  contracting  officers  faced  a  number 
of  time  pressures  which  were  not  so  evident  in  the  pricing 
office.  Contracting  officers  are  constantly  under  pressure 
from  the  customer  to  get  a  contract  awarded  as  soon  as  pos¬ 
sible.  While  pricers  arc  faced  with  time  standards  for 
performing  a  pricing  task,  the  pressures  are  not  as  intense. 
Pricers  stated  that  the  combination  of  the  need  to  get  the 
job  done  quickly  and  the  fact  that  pricing  was  just  one  of 
many  tasks  for  the  contracting  officers  greatly  contributed 
to  the  poor  quality  of  pricing  actions  performed  by  the  con¬ 
tract  ing  off  icers . 

Experience  and  educational  background  were  also  cited 
as  important  factors.  Experience  was  defined  as  the  amount 
of  time  spent  in  the  area  of  pricing.  The  pricers  stated, 
in  general,  that  since  pricing  was  all  they  did  and  since 


they  did  it  on  a  regular  basis,  they  were  naturally  better 
at  it.  Conversely,  since  the  contracting  officers  performed 
pricing  only  on  an  intermittant  basis,  they  were  not  as 
skilled  in  the  area.  The  educational  background  of  the  two 
groups  was  also  mentioned  a  number  of  times.  Pricers,  for 
the  most  part,  had  a  strong  background  in  accounting  and 
financial  management ,  and  stated  that  this  background  was 
necessary  to  understand  the  complex  accounting  systems  of 
most  defense  contractors.  For  instance,  in  one  AFLC  office 
of  the  fourteen  price  analysts,  all  had  an  accounting  degree, 
three  had  Masters  Degrees  in  Accounting,  and  one  had  a  CPA. 
While  this  is  perhaps  higher  than  the  Air  Force  average  of 
60  percent  accounting  degrees  in  pricing,  it  shows  the  depth 
of  accounting  knowledge.  In  the  contracting  offices  on  the 
other  hand,  only  about  40  percent  of  the  professional  per¬ 
sonnel  have  degrees  and  many  of  the  degrees  are  in  areas  not 
related  to  accounting  (15:121).  The  pricers  felt  that  this 
lack  of  an  accounting  background  was  of  critical  significance 
to  the  quality  of  the  pricing  performed. 

While  technical  training  was  also  cited  as  a  factor 
which  contributed  to  the  different  qualitv  of  pricing  actions, 
this  factor  was  considered  somewhat  less  important  than 
education  and  experience.  Further,  the  pricers  stated  that 
technical  training  by  itself,  without  the  prior  educational 
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background,  and  without  continual  experience  in  pricing, 
would  not  suffice  to  make  for  a  quality  pricer. 

In  the  discussions  of  these  four  factors  of  educa¬ 
tion.  experience,  training  and  focus,  the  pricers  contin¬ 
ually  mentioned  the  "skill"  of  the  pricer.  This  was  usually 
defined  as  a  "knack"  to  do  pricing.  While  it  was  related 
somewhat  to  the  four  factors,  it  seemed  to  be  more  related 
to  a  natural  ability  to  do  the  pricing  and  a  natural  liking 
of  the  pricing  job.  A  frequent  comment  was  that  all  pricers 
had  the  "knack"  since  those  who  didn't  had  long  left  the 
pricing  field.  However,  since  only  a  small  portion  of  the 
contracting  officers'  jobs  involved  pricing,  most  of  the  COs 
did  not  have  this  "knack"  for  pricing.  This  seems  to  indi¬ 
cate  that  the  pricers  feel  that  even  with  increased  training, 
contracting  officers  would  not  improve  their  pricing  skills. 

Question  <5.  After  discovering  the  reasons  why  pricers  felt 
they  performed  a  better  job,  we  asked  if  the  pricers  had  any 
evidence  to  support  their  assertion.  The  purpose  of  this 
question  was  to  determine  if  the  pricers  could  back  up  their 
answer  to  question  A. 


QUESTION  5:  DO  YOU  HAVE  ANY  EVIDENCE  OR  INFORMATION  WHICH 
COULD  BE  USED  TO  SUPPORT  YOUR  ANSWER  TO  QUESTION  4? 


RESPONSE 

AFLC 

AFSC 

COMBINED 

Reviews  of  PNMs 

79* 

83* 

82* 

Subjective  feelings 
(no  evidence) 

U 

0 

5 

Pricing  saves  more 

7 

17 

13 

We  found  some  support  of  the  pricers'  responses  to 
question  #4.  Most  of  the  pricers  stated  that  reviews  of  the 
PNM's  of  contracts  priced  by  the  contracting  officers  revealed 
that  the  CO ' s  were  not  doing  a  good  job  of  performing  the 
pricing  task.  Most  pricers  felt  that  based  upon  the  PNMs, 
the  COs  were  missing  essential  cost  areas,  were  not  nego¬ 
tiating  "hard  enough",  and  were  not  documenting  the  file 
properly.  However,  since  the  pricers  review  only  the  PNMs 
and  did  not  review  the  complete  file,  it  could  be  argued  that 
perhaps  only  the  documentation  was  poor  and  not  the  actual 
pricing  task.  A  few  of  the  pricers  stated  that  on  the  aver¬ 
age  pricers  saved  the  Government  10-15  percent  more  than  did 
the  contracting  officers.  However,  no  evidence  was  given  to 
support  this  claim. 

Detailed  Questions 

In  the  first  five  questions  discussed  above,  we 
attempted  to  develop  background  information  and  a  basis  for 


further  questioning.  The  last  five  questions  dealt  directly 
with  the  effectiveness  of  the  dollar  threshold  and  on  ways 
that  the  decision  criteria  might  be  improved.  Our  sixth 
question  dealt  with  the  effectiveness  of  the  dollar  thres¬ 
hold.  Our  purpose  was  to  determine  if  the  pricers  felt 
that  the  threshold  adequately  identified  those  contracts 
which  required  the  special  "knack"  of  pricers  discussed  in 
question  4. 

Question  4 6.  QUESTION  6:  HOW  EFFECTIVE  DO  YOU  THINK  THE 
COLLAR  THRESHOLD  IS  AS  A  DECISION  RULE? 


RESPONSE 

AFLC 

AFSC 

Threshold  is  effective 
as  is 

351 

6% 

Threshold  concept  is 
effective  but  amount 
should  be  raised 

29 

0 

TOTAL  EFFECTIVE 

64* 

61 

Threshold  is  not 
effective  at  all 

36 

76 

Threshold  is  not 

effective  but  is  only 
thing  practicable 

0 

18 

TOTAL  NOT  EFFECTIVE 

361 

94% 

The  responses  reveal  a  marked  difference  between 
AFLC  and  AFSC.  The  difference  is  so  great  that  any  combined 
percentages  would  be  meaningless,  and  therefore  were  not 


computed.  In  AFSC,  94  percent  felt  that  the  dollar  thres¬ 
hold  was  not  effective.  Most  of  the  AFSC  pricers  stated 
that  to  really  identify  those  contracts  needing  the  special 
skills  of  pricers  would  require  review  of  the  contract  and 
the  consideration  of  a  number  of  factors  in  addition  to  the 
dollar  value.  However,  a  number  of  these  pricers  added 
that  while  the  dollar  threshold  was  not  totally  effective 
it  was  the  only  thing  practicable.  The  general  attitude 
was  that  any  other  method  would  require  an  in-depth  ana¬ 
lysis  of  the  contract  prior  to  making  a  decision  and  that 
this  would  require  too  much  time  and  effort.  An  interesting 
note,  however,  is  that  based  upon  the  responses  to  ques¬ 
tions  2  and  3.  this  in-depth  analysis  is  already  being  per¬ 
formed  on  those  contracts  which  must  be  returned  to  the 
contracting  officer  due  to  workload  problems. 

In  AFLC,  on  the  other  hand,  the  vast  majority 
stated  that  the  dollar  threshold  was  very  effective.  A 
few,  but  not  a  large  number,  suggested  raising  the  limit  of 
the  threshold.  This  difference,  we  feel,  can  be  attributed 
to  a  difference  in  philosophy  between  the  two  commands.  As 
discussed  earlier,  AFLC  has  one  threshold  for  all  of  its 
offices,  while  AFSC  has  a  number  of  different  thresholds 
set  by  the  individual  offices.  Further,  AFLC  buys  rela¬ 
tively  standard  off-the-shelf  type  items,  while  AFSC  buys  a 
number  of  different  types  of  items  ranging  from  R&D  to 
production  items. 


Question  ?7,  >8,  »9.  After  discussing  the  effectiveness  of 
the  threshold  we  asked  the  interviewees  to  provide  alterna¬ 
tive  factors  which  they  would  use  if  they  were  free  to 
determine  their  own  decision  rule.  Surprisingly,  even 
though  the  majority  of  AFLC  pricers  felt  that  the  dollar 
threshold  was  effective,  they  still  suggested  a  number  of 
other  factors  which  they  would  use  given  the  opportunity. 

Our  purpose  here  was  to  determine  what  factors  are  con¬ 
sidered  important  in  the  pricing  decision.  We  also  wanted 
to  obtain  a  ranking  of  the  factors  and  an  operational  defi¬ 
nition  for  each  factor.  Questions  *7,  *8,  and  #9  were  used 
together  to  obtain  this  information.  The  interviewees  were 
asked  to  think  of  all  possible  factors,  define  each  one.  and 
to  select  and  rank  the  five  most  important. 

QUESTION  #7:  IF  YOU  COULD  RECOMMEND  A  METHOD  WHICH  CON¬ 
SIDERS  FACTORS  OTHER  THAN  DOLLAR  THRESHOLD.  WHAT  FACTORS 
WOULD  YOU  RECOMMEND? 

QUESTION  *3:  PLEASE  DEFINE  EACH  OF  THE  FACTORS  LISTED 
ABOVE.  MAKE  YOUR  DEFINITIONS  AS  SPECIFIC  AS  POSSIBLE. 

FOR  INSTANCE.  IF  YOU'VE  LISTED  TYPE  OF  CONTRACT  AS  A 
FACTOR.  YOUR  DEFINITION  MIGHT  BE  "AS  DEFINED  BY  DAR . ” 

QUESTION  *9:  OF  THE  FACTORS  LISTED  ABOVE.  SELECT  THE  FIVE 
(5)  YOU  CONSIDERED  TO  BE  THE  MOST  IMPORTANT.  INDICATE  WHY 
YOU  FEEL  THESE  ARE  IMPORTANT  FACTORS  AND  RANK  THEM  IN  ORDER 
OF  IMPORTANCE  WITH  THE  MOST  IMPORTANT  RANKED  *1 . 
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Identification  and  Definition  of  Factors 

The  pricers  cited  eleven  (11)  factors  as  being 
important  in  the  decision  to  perform  price  analysis  on  a 
particular  contract.  These  factors  were  discussed  in  detail 
in  order  to  obtain  a  clear,  concise,  operational  definition 
which  did  not  require  any  subjective  judgement.  However, 
on  some  factors  such  as  Contracting  Officer  Skill,  we  were 
not  able  to  do  this.  The  definition  put  forth  by  the 
majority  of  the  pricers  was  subjective  and  judgemental  and 
all  attempts  to  definitize  it  failed.  The  eleven  factors 
and  their  definitions  are  as  follows: 


FACTOR  DEFINITIONS 

CONTRACTING  OFFICER  SKILL  How  good  of  a  pricer  the  CO 

who  would  be  assigned  the 
action  is.  According  to  the 
pricers,  this  does  not  relate 
to  the  number  of  years  of 
experience  of  the  CO,  or  the 
number  of  pricing  courses 
taken.  Rather  it  depends  upon 
whether  the  CO  has  the  "knack" 
for  pricing.  In  each  pricing 
office  the  pricers  felt  that 
they  could  identify  those  COs 
who  possessed  this  "knack". 

TYPE  OF  CONTRACT  Reimbursement  arrangement  as 

defined  by  DAR  (includes  FFP, 
FPIF.  CPFF ,  CPIF,  CPAF ,  CR) . 

NATURE  OF  BUY  Type  of  action  based  upon  the 

history  of  the  buy.  Whether 
the  item  is  an  initial  buy  or 
a  follow  on  effort  for  an  item 
purchased  previously. 


FACTOR 


DEFINITIONS 


AMOUNT  OF  COMPETITION  Whether  effort  is  sole  source 

or  competitive. 

CONTRACTOR  The  difficulty  of  dealing  with 

the  particular  contractor 
involved.  It  is  a  function  of 
the  contractor's  accounting 
system,  past  performance  dur¬ 
ing  negotiations,  and  complex¬ 
ity  of  proposals. 

AMOUNT  OF  DATA  AVAILABLE  Amount  and  quality  of  data 

available  for  analysis. 

Depends  upon  quality  and 
findings  of  the  audit  report, 
and  information  in  contrac¬ 
ting  files  for  same  contrac¬ 
tor  or  like  items. 

PERIOD  OF  PERFORMANCE  Length  of  time  from  the  date 

of  award  of  the  contractor  to 
completion  or  delivery  of 
final  item. 

SPECIAL  PROVISIONS  Special  pricing  arrangements 

as  evidenced  by  presence  of 
escalation  clauses,  state 
tax  clauses,  or  milestone-- 
billing  provisions. 

DOLLAR  VALUE  Amount  of  estimated  cost  pro- 

prosa 1 . 


After  each  pricer  identified  and  defined  the 
factors  considered  to  be  important,  he  was  asked  to  select 
and  rank  the  five  that  he  considered  to  be  the  most  critical 
in  the  pricing  decision.  The  rankings  are  shown  on  the  fol¬ 
lowing  tables.  Factors  are  shown  with  a  one  (1)  for  the 
factor  selected  as  most  important  and  a  five  (5)  for  the 
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factors  considered  the  least  important.  Some  interviewees 
identified  less  than  five  factors  and  therefore  ranked  less 
than  five  factors. 

Analysis  of  Rankings 

As  discussed  in  Chapter  2,  our  purpose  in  obtaining 
a  ranking  of  factors  was  to  develop  a  model  based  upon  the 
rankings  and  decision  rules  provided  by  the  interviewees. 
The  decision  rules  were  provided  in  response  to  question 
ten  which  will  be  discussed  later.  However,  prior  to  using 
the  decision  rules  in  the  development  of  our  model,  we 
tested  the  rankings  for  consistency.  If  the  rankings  were 
not  consistent,  their  use  in  the  development  of  a  model 
would  be  questionable.  Therefore,  before  conducting  an 
analysis  of  the  response  to  question  ten.  the  rankings  pro¬ 
vided  in  question  nine  were  analyzed. 

Consistency  was  tested  through  the  use  of  the  Ken¬ 
dall  W  test  described  in  detail  in  Chapter  2.  In  perfor¬ 
ming  the  test,  we  used  the  standard  SPSS  subprogram  NPAR 
test  package.  Therefore,  test  statistics  were  not  computed 
manually;  however,  a  listing  of  the  applicable  formulae  for 
the  tests  is  shown  in  Appendix  H.  In  using  this  test,  we 
established  a  data  file  of  the  rankings  provided  by  the 
pricers.  A  separate  file  was  established  for  each  command 
as  well  as  a  file  for  all  of  the  responses.  Each  file 
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contained  the  number  of  cases  (interviews)  in  the  file, 
and  consisted  of  11  variables,  one  for  each  factor.  The 
rankings  (1-5)  for  each  factor  were  recorded  in  the  data 
file  whenever  the  factor  was  ranked  for  a  specific  case. 

If  the  factor  was  not  ranked,  a  value  of  six  was  given  to 
the  variable.  For  example,  the  data  file  for  AFLC  inter¬ 
viewee  number  one  (1)  was  as  follows: 

64366162666 

A  complete  listing  of  all  of  the  data  files  the  programs 
used  to  run  the  Kendall  test  and  the  program  output  are 
contained  in  Appendices  E,  F,  and  C. 

In  performing  the  Kendall  test,  we  realized  that 
since  in  every  case  at  least  seven  out  of  eleven  variables 
would  be  ranked  as  a  six  (showing  non-selection)  that  the 
test  was  actually  testing  the  consistency  of  those  not 
ranked.  However,  since  all  cases  contained  the  same  11 
factors,  if  the  results  showed  consistency  of  the  factors 
not  ranked,  consistency  of  the  factors  ranked  could  be 
assumed . 

The  results  of  the  Kendall  W  test  are  as  follows. 


AFL 

U 

Ch i -Square 

Significance  Level 

AFLC 

.2692 

37.6891 

.0000 

AFSC 

.2692 

48.4485 

.0000 

COMBINED 

.1992 

63.7479 

.0000 
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In  Che  Kendall  test  a  W  score  of  close  to  one 
usually  indicates  consistency,  so  it  may  appear  from  the 
results  above  that  there  is  no  consistency  in  the 
responses.  However,  the  significance  level  is  also  a 
measure  of  cons i s tency- -one  which  takes  into  consideration 
both  the  W  score  and  the  Chi-square  statistic.  The  Chi- 
square  statistic  is  a  measure  that  when  tested  with  the 
degrees  of  freedom  indicates  the  probability  that  our  null 
hypothesis  (no  agreement  exists)  can  be  rejected.  Due  to 
the  high  Chi-square  statistic  in  all  three  tests,  which 
allows  for  Che  rejection  of  the  null  hypothesis  at  signifi¬ 
cance  levels  of  less  than  .001,  we  were  able  to  conclude 
that  the  consistency  indicated  by  the  Kendall  test  was  due 
to  more  than  pure  chance  (17:237).  On  this  basis,  we  con¬ 
cluded  that  there  was  consistency  in  the  five  factors 
selected  most  often,  but  we  were  not  able  to  conclude  con¬ 
sistency  in  the  ordering  or  ranking  of  the  five  factors. 

Therefore,  in  order  to  develop  an  ordering  of  the 
five  factors,  were  performed  two  tests.  One  was  the  com¬ 
putation  of  weighted  averages  based  upon  the  ranks  provided 
during  the  interviews,  and  the  second  was  a  review  of  the 
Kendall  mean  ranks.  Both  of  these  tests  produced  the  same 
results  and  showed  the  ordering  of  the  top  five  factors  to 
be  as  listed  below. 
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AFLC 


Nature  of  buy 
Amount  of  data 
Type  of  contract 
Contracting  officer  skill 
Complexity 


AFSC 

Type  of  contract 
Contractor 

Contracting  officer  skill 
Complex i ty 
Nature  of  buy 

As  can  be  seen  from  the  rankings  above,  not  only 
are  the  factors  which  were  considered  important  consistent, 
the  selection  of  the  top  five  is  also  very  consistent 
between  the  commands.  Four  out  five  of  the  factors  are 
the  same  in  both  groups,  and  the  factors  which  are  different 
are  quite  related.  The  amount  of  data  available  is,  in  many 
cases,  related  to  whom  the  contractor  is.  Because  the 
rankings  by  command  were  so  similar,  we  felt  that  further 
analysis  and  model  development  could  be  done  on  the  basis 
of  a  combined  ranking  of  the  results  of  the  two  commands. 

The  combined  ranking  was  computed  in  the  same  manner  as  the 
individual  rankings  and  produced  the  following  five  factors 
in  order  of  importance. 
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Type  of  contract 
Nature  of  buy 
Contracting  officer  skill 
Complex i ty 
Contractor 

We  find  this  final  ranking  interesting  from  two 
standpoints.  First,  complexity  is  ranked  last  of  the  five 
factors.  This  factor  was  mentioned  in  all  of  our  prelimi¬ 
nary  interviews  and  was  constantly  discussed  during  the 
data  collection  interviews.  Yet,  when  ranked  against  other 
factors,  the  pricers  found  complexity  to  be  relatively  un¬ 
important.  The  second  interesting  feature  is  that  dollar 
value  does  not  even  make  it  to  the  top  five.  This  seems 
to  be  inconsistent  with  the  responses  to  question  ten. 

Quest  ion  *  10 .  In  question  #10  we  asked  the  pricers  to 
develop  decision  rules  for  the  factors  which  they  felt 
were  the  most  significant.  We  specifically  asked  for  a 
decision  rule  which  could  stand  alone;  in  other  words,  one 
which  could  be  used  without  the  consideration  of  any  other 
factor.  However,  the  pricers  for  the  most  part  would  not 
commit  themselves  to  rules  which  did  not  in  some  way  con¬ 
sider  dollar  value.  The  pricers  stated  that  any  decision 
rule  had  to  be  used  in  conjunction  with  the  dollar  value 
of  the  proposal.  The  decision  rules  proposed  for  the  top 
five  factors  on  an  Air  Force  wide  basis  are  as  follows. 


QUESTION  10:  FOR  EACH  OF  THE  FIVE  FACTORS  SELECTED  ABOVE. 
PROVIDE  A  DECISION  RULE  WHICH  YOU  FEEL  WOULD  BE  APPROPRIATE 
TO  USE  IN  DECIDINC  IF  A  PRICING  ACTION  SHOULD  BE  ACCEPTED 
BY  THIS  OFFICE.  FOR  EXAMPLE,  IF  ONE  OF  YOUR  FACTORS  IS 
TYPE  OF  CONTRACT,  YOUR  DECISION  RULE  MAY  BE  "ALL  COST  TYPES 
SHOULD  COME  HERE.  ALL  FIXED  PRICE  SHOULD  CO  TO  CONTRACTING 
OFFICER." 


FACTOR 

TYPE  OF  CONTRACT 

COMPLEXITY 

CO  SKILL 

NATURE  OF  BUY 


RULE 

a  .  FFP  &  incentive 
to  pricing 

Cost  to  CO 

b.  FFP  to  CO 

all  ether  types 
to  pricing 

Complex  go  to 
pricing 

Non-complex  go  to 
CO 

CO  is  competent--CO 
CO  is  not--pricing 

Follow  or  standard 
items  to  CO 


291  781  567. 

14  22  19 

36  67  53 

21  44  34 

43  36  34 


Non-standard  to 
pr icing 

CONTRACTOR  Accounting  system  43  44 

is  difficult,  poor 
history  for  nego- 
t iat ions 

complex  proposal-- 
go  to  pricing 

Non-d ifficult.  etc. 
go  to  CO 
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Model  Development 

As  discussed  earlier  and  in  Chapter  2,  we  intended 
to  develop  decision  rules  which  (1)  could  stand  alone  and 
be  used  as  the  basis  for  decision  tree  analysis,  and  (2) 
reflected  agreement  of  at  least  51  percent  of  the  pricers. 
However,  this  was  not  feasible.  First,  only  two  of  the 
decision  rules  reflected  agreement  of  at  least  51  percent; 
and  second,  four  out  of  the  five  decision  rules  not  only 
required  subjective  evaluations,  they  also  contained  the 
caveat  that  they  were  to  be  used  in  conjunction  with  some 
dollar  level. 

Therefore,  since  the  use  of  decision  tree  analysis 
was  not  appropriate,  we  took  a  different  approach  in 
developing  our  model.  We  included  all  five  decision  rules 
which  related  directly  to  the  top  five  factors  in  our  model, 
giving  equal  weight  to  each  rule.  In  other  words,  no 
attempt  was  made  to  weight  the  factors  to  reflect  the 
ordering  or  ranking.  In  making  a  decision  with  the  model, 
our  model  called  for  a  separate  decison  to  be  made  on  the 
basis  of  each  decision  rule.  The  decision  (either  for 
pricing  or  for  the  CO)  which  received  the  majority  of  the 
decisions  made  on  the  basis  of  each  decision  rule  would 
reflect  the  overall  decision  of  the  model.  For  example, 
if  three  of  the  decision  rules  resulted  in  a  decision  that 
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the  action  should  be  priced  by  the  CO,  and  two  of  the  rules 
resulted  in  a  decision  for  the  pricing  staff,  then  the  over¬ 
all  decison  would  be  to  have  the  action  priced  by  the  CO. 

Test  of  the  Model 

To  test  the  model  we  abstracted  information  per¬ 
taining  to  the  decision  rules  from  the  selected  contract 
file.  However,  we  found  that  we  could  abstract  information 
on  only  three  of  the  five  decision  rules.  Information  was 
not  available  to  make  a  decision  on  the  CO  skill  or  the 
amount  of  data.  Therefore,  these  decision  rules  were  not 
used  in  this  instance.  The  results  of  our  decisions  based 
upon  the  model  (reflecting  only  three  factors)  are  shown  in 
the  following  tables. 

The  results  indicate  that  the  inclusion  of  factors 
other  than  dollar  value  in  the  decision  making  process 
produces  decisions  different  than  those  produced  by  use  of 
the  dollar  threshold  alone  According  to  our  results,  19 
of  the  47  contracts  tested  (40.4  percent)  were  priced  by 
the  wrong  office.  Of  the  24  contracts  originally  sent  to 
pricing.  14  should  have  been  handled  by  the  contracting 
office;  of  the  23  contracts  originally  handled  in  the  con¬ 
tracting  office,  five  should  have  been  sent  to  pricing. 

By  command,  the  results  were  as  follows. 
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Table  3 


Model  Res u 1 t s - -AFLC 


Contract  Number 

Threshold  Decision 

Model  Decision 

1 

CO 

CO 

2 

P 

CO 

3 

CO 

CO 

U 

P 

CO 

5 

CO 

CO 

6 

P 

CO 

7 

P 

CO 

8 

CO 

CO 

9 

P 

CO 

10 

P 

CO 

11 

CO 

p 

12 

P 

CO 

13 

CO 

CO 

U 

CO 

CO 

15 

CO 

CO 

16 

CO 

CO 

17 

CO 

CO 

18 

P 

CO 

19 

CO 

CO 

20 

p 

CO 

P  -  Pricing 

CO  ■  Contracting  Officer 
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Table  4 


Model  Resul ts- -AKSC 


Contract  Number  Threshold  Decision  Model  Decision 


1 

P 

o 

L 

P 

3 

P 

u 

P 

5 

P 

6 

P 

7 

P 

8 

P 

9 

P 

10 

P 

11 

CO 

12 

CO 

13 

CO 

14 

CO 

15 

CO 

16 

CO 

17 

CO 

18 

CO 

19 

p 

20 

p 

21 

CO 

22 

CO 

23 

CO 

24 

CO 

25 

p 

26 

p 

26 

Pricing 

p 

CO  -  Contracting  Officer 


P 

P 

P 

P 

P 

P 

CO 

P 

CO 

P 

CO 

CO 

P 

CO 

CO 

P 

CO 

CO 

P 

CO 

P 

P 

CO 

CO 

P 

CO 

CO 
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AFLC 


Of  9  originally  handled  in  pricing. 

9  should  have  been  handled  by  the  CO 

Of  11  originally  handled  by  the  CO, 

1  should  have  been  handled  by  pricing 

AFSC 

Of  15  originally  handled  in  pricing, 

5  should  have  been  handled  by  the  CO 

Of  12  originally  handled  by  the  CO. 

9  should  have  been  handled  by  pricing 

These  results  indicate  a  number  of  things.  First, 
the  use  of  the  model  does  appear  to  change  the  outcome  of 
pricing  decisions.  Second .  while  the  use  of  the  model 
results  in  no  net  change  *o  the  number  of  contracts  priced 
by  pricing  personnel  in  AFSC,  it  does  produce  a  significant 
change  for  AFLC  personnel.  The  use  of  the  model  results  in 
only  one  out  of  twenty  AFLC  contracts  being  priced  in  the 
pricing  office,  with  all  of  the  remaining  being  priced  by 
the  CO.  This  suggests  that  the  pricing  offices  in  AFLC 
could  be  descreased  significantly. 
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While  the  use  of  this  model  does  result  in  dif¬ 
ferent  decisions  than  the  use  of  the  dollar  threshold 
alone,  its  use  and  therefore  the  results  are  subject  to 
a  number  of  limitations.  First,  while  the  model  developed 
based  upon  the  interviews  calls  for  the  inclusion  of  five 
decision  rules,  only  three  were  included  in  the  test  model 
due  to  lack  of  information  on  the  other  two.  The  inclu¬ 
sion  of  the  other  two  rules  could  alter  the  results  of  the 
test.  Second,  the  use  of  the  model  calls  for  subjective 
and  judgemental  evaluations  which  should  be  made  by  pricing 
experts.  While  we  did  use  the  information  provided  by 
pricing  experts  to  make  our  decisions,  we  are  not  pricing 
experts.  The  results  of  the  test  may  have  been  different 
had  the  decisions  been  actually  made  by  the  pricing 
experts.  Third,  no  weighting  was  assigned  to  any  factor; 
all  were  given  equal  consideration.  If  a  weighting  was 
assigned  based  upon  the  ordering  of  the  top  five  factors, 
the  test  results  could  have  been  different.  And  fourth, 
data  from  contract  files  were  abstracted  from  files  at 
only  one  location  for  AFLC  (SAALC)  and  from  only  one 
major  AFSC  office  (ESD)  and  one  non-major  AFSC  office 
(R&D  Contracting).  Had  contracts  been  selected  from  addi¬ 
tional  bases,  the  test  results  might  have  been  different. 


Chapter  A 


FINDINGS  AND  RECOMMENDATIONS 

This  chapter  discusses  our  overall  findings  and 
how  they  relate  to  the  research  objectives  and  questions, 
our  recommendations  for  a  pricing  policy,  other  related 
observations,  and  our  r ecommenda t ion  for  future  study. 


Achievement  of  Ob jec t i ves /Ques t i ons 

As  a  result  of  this  research  effort,  all  of  our 
objectives  were,  although  to  varying  degrees,  met.  Our 
first  objective  was  to  determine  what  criteria  are  per¬ 
ceived.  by  the  pricing  community,  to  influence  the  need 
for  pricing  staff  action.  This  study  revealed  that  11 
factors,  listed  below,  were  considered  important. 

CO  experience 
Type  of  contract 
Nature  of  buy 
Type  of  instrument 
Complex i t  v 

Degree  of  competition 
Contractor 

Amount  and  completeness  of  field  reports 
Performance  period 
Special  provisions 
Dollar  value 

We  believe  that  these  results  fully  meet  the  first 
research  objective.  However,  we  feel  that  we  have  only 
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partially  met  the  other  two  objectives  and  that  these  areas 
warrant  further  study. 

The  second  objective  was  to  develop  a  decision 
model,  based  upon  a  decision  tree  analysis,  that  would 
predict  when  pricing  staff  action  should  be  performed. 

While  we  did  develop  a  model,  it  was  not  based  upon  a 
decision  tree  analysis.  Rather  the  model  was  simply  a 
statement  that  the  top  five  factors  should  be  used  in  the 
decision  making  process  which  employs  a  "majority  rule" 
decision.  This  model  can,  however,  be  used  for  predictive 
purposes  if  the  pricing  experts  use  their  subjective 
judgement  in  applying  the  model. 

The  third  objective  was  to  test  the  model  to  deter¬ 
mine  if  its  use  resulted  in  a  different  decision  than  the 
use  of  the  dollar  threshold.  While  we  tested  the  model  in 
this  manner  and  determined  that  the  results  were  different, 
the  test  is  suspect  for  two  reasons:  first,  the  model 
called  for  subjective  evaluations  by  pricing  experts,  and 
we  are  not  pricing  experts.  Second,  because  of  the  unavail¬ 
ability  of  information,  we  were  able  only  to  make  judgements 
on  three  of  the  five  factors.  Therefore,  two  factors  which 
should  have  been  included  were  not. 

Our  research  pro.  ded  answers  to  all  of  the  research 
questions.  Our  first  question  asked  how  the  current  method 
could  be  improved.  Our  findings  indicated  that  the  current 
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method  could  be  improved  by  the  inclusion  of  other  criteria, 
in  addition  to  dollar  value,  in  the  decision  making  process. 
Inclusion  of  additional  crireria  would  alleviate  the  fol¬ 
lowing  two  deficiencies  noted  in  the  current  method. 

1.  The  present  method  is  not  effective  in  identi¬ 
fying  the  contracts  which  require  the  specialized  expertise 
of  the  pricing  staff. 

2.  The  threshold  rule,  in  practice,  is  not  enough 
of  a  basis  for  a  decision. 

The  second  question  dealt  with  determining  what 
factors  the  experts  considered  important  in  the  pricing 
decision.  Our  research  identified  the  11  factors  listed 
earlier. 

The  third  question  focused  on  the  difference  between 
the  skill  of  the  pricers  versus  the  skill  of  the  COs .  Our 
research  indicated  that  the  "skill"  is  really  a  knack  or 
ability  to  perform  pricing  which  is  related  to  education, 
experience  and  training.  Further,  the  ability  to  use  this 
knack  depends  upon  the  focus  or  attitude  of  the  person  per¬ 
forming  the  analysis. 

The  fourth  research  question  asked  for  a  defini¬ 
tion  of  complexity.  Our  interviews  revealed  that  most 
pricers  defined  complexity  from  a  technical  standpoint. 
Complexity  involved  the  state-of-the-art.  the  level  of 

technology,  and  the  existence  of  special  technical  features 
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such  as  Reliability  Improvement  Warranty  (RIW)  or  Design 
to  Cost  (DTC) . 

The  final  question,  and  perhaps  the  most  important, 
asked  if  the  use  of  criteria  other  than  dollar  threshold 
would  result  in  decisions  which  would  provide  a  more  effi¬ 
cient  use  of  resources,  i.e.,  the  pricing  staff.  Based  upon 
our  model's  test,  we  answer  this  question  with  a  qualified 
"yes".  Since  the  use  of  the  model  resulted  in  different 
decisions,  and  since  the  model  was  based  upon  criteria 
which  pricing  experts  indicated  were  important,  we  conclude 
that  the  model's  decisions  provide  a  more  efficient  use  of 
resources  . 

Fi nd i ngs 

With  most  efforts,  research  is  undertaken  with  the 
intent  of  having  the  findings  either  prove  or  disprove 
certain  a  priori  expectations.  This  effort  was  an  exception 
in  that  we  had  no  expectations  to  prove  or  disprove.  We 
did  however  believe,  with  some  justification,  that  other 
criteria  besides  dollar  value  could  be  used  in  the  pricing 
decision.  Our  efforts  were  directed  to  determining  if  our 
beliefs  were  merited.  In  Chapter  3,  we  described  the 
identified  criteria,  and  the  decision  model  developed  from 
them.  Throughout  that  chapter,  we  either  directly  referred 
to  or  alluded  to  several  overall  findings  of  this  effort. 

For  purposes  of  summation,  these  findings  can  be  divided 
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into  major  and  minor  categories.  In  the  major  category 
we  included: 


1.  AFLC  appears  to  be  pricing  more  proposals  than 
it  needs  to. 

2.  AFSC  while  maintaining  the  same  relative  work¬ 
load,  requires  a  shift  in  emphasis  on  which  proposals 
should  be  priced. 

In  the  minor  category  we  included: 

1.  Most  pricing  divisions  are  implicitly  using 
the  identified  criteria.  However,  this  is  being  done  only 
when  workload  becomes  a  factor  and,  with  the  exception  of 
two  pricing  offices,  no  written  policy  on  the  use  of  the 
criteria  is  available. 

In  addition  to  these  findings,  were  received  several 
observations/comments  from  those  we  interviewed.  The  fol¬ 
lowing  is  a  list  of  observations  we  derived  from  the  com¬ 
ments  of  the  interviewees. 

1.  COs ,  as  a  whole,  do  not  possess  the  expertise 
to  analyze  many  of  the  contracts.  This  opinion  seemed 
more  prevalent  within  AFLC  than  AFSC. 

2.  The  five  identified  criteria  should  be  used  as 
a  basis  for  making  the  pricing  decision,  in  conjunction 
with  a  dollar  threshold  level.  While  this  is  somewhat 
similar  to  current  practice,  the  major  difference  is  that 
all  proposals  above  the  threshold  would  always  be  evaluated 
not  just  when  workload  is  a  factor. 

3.  Each  pricing  office  should  establish  a  written 
policy  for  its  decision  criteria  and  should  coordinate  that 
policy  with  the  contracting  office. 

4.  When  efforts  must  be  returned  to  the  CO  for 
action,  a  partial  analysis  should  be  accomplished.  The 
procedure  currently  used  at  SAMSO  could  be  used  as  a  guide¬ 
line. 


5.  AFLC  should  critically  examine  its  use  of 
$100,000  as  a  threshold  level.  The  level  should  be 
periodically  examined  and  possibly  tied  to  an  index. 
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6.  The  $100,000  threshold,  established  by  PL 
87-653,  was  set  in  1962.  It  is  outdated  and  should  be 
raised  to  parallel  the  rise  in  the  consumer  price  index. 
The  time  required  to  analyze  a  proposal  has  not  changed 
over  the  years;  however,  the  amount  of  money  saved  per 
effort  has  declined  because  the  contractors  are  more 
aware  of  the  government's  requirements  for  proposals  and 
thus  submit  better  proposals.  In  addition,  the  salary 
of  the  average  analyst  has  risen  since  1962.  As  a 
result  of  the  above  factors,  pricing  of  a  proposal  in  the 
neighborhood  of  $100,000  is  no  longer  cost  effective. 

7.  The  pricing  office  should  receive  all  pro¬ 
posals  above  a  minimum  threshold.  For  proposals  above 
this  minimum  but  below  some  higher  threshold,  the  pricing 
office  should  compare  the  proposal  to  the  field  audits. 

If  the  differences  do  not  exceed  a  specified  percent  the 
proposal  should  be  returned  to  the  CO,  otherwise,  the  pri¬ 
cing  office  should  conduct  the  analysis. 

8.  The  threshold  level  should  be  raised  to 
$500,000  as  recommended  by  Senator  Childs. 

9.  Tie  any  increase  in  the  threshold  level  to  the 
requirement  for  a  contractor  to  file  a  DD  633. 


Significance  of  Findings 

Comparing  the  results  of  the  model's  tests  to  our 
a  priori  expectations,  provided  some  unexpected  results. 

As  stated  in  the  problem  statement,  in  Chapter  1,  we 
believed  that  criteria  existed  which  were  inherent  in  all 
contracts  and  which  could  be  used  to  determine  which  pro¬ 
posals  required  specialized  analysis  by  the  pricing  office. 
We  further  believed  that  by  identifying  these  factors, 
proper  emphasis  could  be  placed  on  those  proposals  which 
required  the  expertise  of  the  pricing  staffs.  Thus,  we 
expected  the  test  results  to  show  a  change  in  emphasis  on 


who  performs  the  analysis,  but  no  drastic  change  in  the 
number  of  proposals  analyzed  by  each  divisi  ■ 

Test  results  of  the  model  on  contracts  sampled  from 
AFSC  were  just  as  expected.  As  previously  mentioned,  of 
the  27  AFSC  contracts  sampled.  15  were  originally  analyzed 
by  pricers  while  12  were  analyzed  by  COs .  After  applica¬ 
tion  of  the  model,  33  percent  should  be  analyzed  by  pricers 
while  67  percent  should  be  analyzed  by  COs.  Although 
there  was  no  drastic  change  in  the  number  each  division 
should  analyze,  the  emphasis  was  redirected  as  evidenced 
by  the  nine  contracts  which  changed  divisions.  Unlike  the 
AFSC  results,  quite  different  results  were  found  with  the 
AFLC  sample  contracts. 

Model  results  of  the  AFLC  sampled  contract  indi¬ 
cated  a  very  significant  change  in  workload.  Of  the  20 
contracts  from  an  ALC,  half  were  originally  analyzed  by 
pricers  and  half  by  COs.  The  model  results  indicated 
that  only  one  needed  the  pricing  division's  expertise. 

This  result,  if  replicated,  could  prove  to  have  a  far 
reaching  effect  upon  the  workforce  makeup  of  this  ALC. 

In  fact,  if  the  other  ALCs  were  sampled,  similar  results 
would  necessitate  a  command-wide  evaluation  by  AFLC  of 
its  workforce  at  each  ALC. 


As  a  result  of  this  effort,  we  have  several  recom¬ 
mendations  which  we  believe  would  inhance  and  refine  the 
results  already  obtained.  These  recommendations  can  be 
divided  into  two  areas:  (l)  recommendations  concerning 
the  criteria  used  in  the  model,  and  (2)  recommendations 
concerning  testing  the  model. 

Model  criteria.  Currently  the  model's  decision  of  who  per¬ 
forms  the  pricing  action  is  based  upon  a  simple  "majority 
rule".  This  occurs  because  the  criteria  are  not  weighted. 
The  decision  can  be  refined  by  weighting  the  criteria  in¬ 
cluded  in  the  model.  The  weighting  will  remove  the  subjec- 
tiveness  of  the  overall  model  decision.'*  This  weighting 
process  can  be  accomplished  in  three  steps. 

The  first  step  would  involve  formally  defining  each 
criteria  identified  in  this  research  effort.  This  is  neces¬ 
sary  so  that  future  interviewees  will  have  a  common  ground 
from  which  to  rate. 

Second,  some  form  of  a  Likert  scale  should  be 
devised  so  that  each  criteria  can  be  weighted  either  against 
the  other  criteria  or  against  overall  importance. 


\some  subjectiveness  would  still  exist  in  some  of 
the  criteria,  for  example,  CO  experience. 
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The  third  step  would  be  to  incorporate  the  first 
two  steps  into  a  questionnaire,  and  to  take  a  larger  sample 
of  the  pricing  community.  In  fact,  a  sample  of  the  entire 
pricing  population  could  be  made  without  difficulty. 

Model  testing.  In  addition  to  enhancing  the  model,  we 
believe  that  further  tests  of  the  model  should  be  performed. 
Further  tests,  with  the  enhanced  model,  should  be  conducted 
to  refute  or  confirm  the  preliminary  findings.  We  firmly 
believe  that  in  light  of  the  surprising  results  from  San 
Antonio  ALC's  sampled  contracts,  contracts  from  all  the  ALCs 
should  be  sampled  and  included  in  the  model’s  tests. 

Other  Related  Findings 

Two  findings  were  revealed  in  areas  related  to 
pricing  but  not  directly  related  to  the  dollar  threshold 
or  other  criteria.  The  first  is  that  the  Copper  Impact 
System  is  not  being  used  by  most  of  the  AFLC  offices  and 
by  a  number  of  the  AFSC  offices.  This  seems  to  be  due  to 
a  lack  of  understanding  of  the  system  and  the  inappropriate- 
ness  of  many  of  the  standard  impact  programs.  The  second 
finding  is  that  pricing  offices  have  a  need  for  someone 
with  both  pricing  and  computer  programming  ability. 

Further  study  in  these  areas  is  warranted. 
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Air  Force  Logistics  Command 


♦Oklahoma  City  Air  Logistics  Center  (ALC) 

♦Ogden  ALC 

♦San  Antonio  ALC 

♦Warner  Robbins  ALC 

♦Sacramento  ALC 

Air  Force  Systems  Command 


Major  Centers 

♦Aeronautical  Systems  Division 

♦Electronic  Systems  Division 

♦Space  and  Missile  Systems  Organization 

Non -Major  Centers 

♦Armament  Development  and  Test  Center 
Air  Force  Flight  Test  Center 
Rome  Air  Development  Center 

♦Directorate  of  Research  and  Development  Contracting 
Air  Force  Contract  Management  Division 

Directorate  of  Contracting,  Space  and  Missile  Technical 
Center 

Arnold  Engineering  Development  Center 

e  1  «*  red  offices  included  in  the  population  are  marked  with 
an  asterisk. 
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APPENDIX  C 

LETTER  TO  INTERVIEWEES 


DEPARTMENT  OF  THE  AIR  FORCE 

t>R  FORCE  INS*'TUTE  Of  TECHNOLOGY  atc 
.".  RIGM*-  PATTERSON  A I R  FQRC  E  S»SE  OHIO  41433 


May  4,  1979 


Dear  : 

You  have  been  selected,  as  a  pricing  expert,  to  participate 
in  our  Air  Force  Institute  of  Technology  (AFIT)  research 
thesis  on  the  effectiveness  of  the  use  of  a  dollar  threshold 
as  the  sole  criterion  for  determining  when  a  proposal  is 
forwarded  to  the  pricing  office  for  analysis.  The  objective 
of  the  research  thesis,  which  is  being  sponsored  by  HQ  USAF, 
is  to  determine  the  feasibility  of  using  other  decision 
criteria . 

Within  the  next  three  weeks,  Capt.  McConnell  or  I  will  contact 
you  and  arrange  a  convenient  time  for  a  telephone  interview 
with  you  on  this  subject.  The  interview  will  last  approximately 
one  hour  and  will  follow  the  attached  interview  guide.  Since 
the  validity  of  our  research  results  will  depend,  in  part, 
on  the  independence  of  the  responses,  please  do  not  discuss 
the  interview  questions  with  others  in  your  office.  Your 
responses  will  be  kept  confidential.  All  responses  will  be 
tabulated  and  reported  by  categories  only  so  that  no  specific 
response  will  be  attributed  to  any  individual. 

The  results  of  the  effort  will  be  forwarded  to  HQ  USAF  for 
possible  use  in  future  pricing  policy  decisions.  Your 
cooperation  and  support  of  this  effort  would  be  greatly 
appreciated . 


MARGARET  A.  MARTINEZ 
AFXT/LSG  Class  79B 
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1.  According  Co  your  Office  Operating  Instruction,  a  dollar 
threshold  is  the  sole  criterion  for  determining  whether  a 
pricing  action  is  accepted  by  this  office. 


a.  In  your  opinion,  what  is  the  basis  for  the  use  of 
the  dollar  threshold  as  the  decision  criterion? 


b.  In  your  opinion,  what  is  the  basis  for  the  selec¬ 
tion  of  the  particular  dollar  value  used  as  the 
threshold? 


2.  Do  you  know  of  any  pricing  actions  accepted  by  this 
office  which  did  not  follow  the  dollar  threshold  rule? 

In  other  words,  contracts  above  the  threshold  being 
returned  to  the  Contracting  Officer,  or  contracts  below 
the  threshold  being  accepted  by  this  office? 

3.  If  your  answer  to  question  #2  is  yes: 

a.  What  percentage  of  the  total  number  of  contracts 
accepted  by  this  office  does  not  follow  the  thres¬ 
hold  rule? 


b.  Who  makes  the  decision  to  accept /ret urn  these 
act  ions  ? 


c.  On  what  basis  is  the  decision  made? 


d.  Is  there  a  written  policy  on  this  practice? 


U.  Do  you  feel  that  there  is  a  difference  in  the  effec¬ 
tiveness  (accuracy,  completeness,  timeliness,  dollar  savings 
to  the  Government)  of  pricing  actions  performed  by  this 
office  as  compared  to  pricing  actions  performed  by  the  Con¬ 
tract  ing  Off icers? 
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5.  Do  you  have  any  evidence  or  information  which  could 
be  used  to  support  your  answer  to  question  #4? 


6.  How  effective  do  you  think  the  dollar  threshold  is  as 
a  decision  rule? 


7.  If  you  could  recommend  a  method  which  considers  fac¬ 
tors  other  than  dollar  threshold,  what  factors  would  you 
recommend?  List  as  many  factors  as  you  can  think  of. 


8.  Please  define  each  of  the  factors  listed  above.  Make 
your  definitions  as  specific  as  possible.  For  instance, 
if  you've  listed  "type  of  contract"  as  a  factor,  your 
definition  might  be  as  defined  by  DAR." 


9.  Of  the  factors  listed  above,  select  the  five  (5)  you 
consider  to  be  the  most  important.  Indicate  why  you  feel 
these  are  important  factors  and  rank  them  in  order  of 
importance  with  the  most  ranked  1. 


10.  For  each  of  the  five  factors  selected  above,  provide 
a  decision  rule  which  you  feel  would  be  appropriate  to 
use  in  deciding  if  a  pricing  action  should  be  accepted  by 
this  office.  For  example,  if  one  of  your  factors  is  type 
of  contract,  your  decision  rule  might  be  "all  incentive 
type  contracts  should  be  accepted  by  this  office;  no  CPFF 
type  contracts  should  be  accepted  by  this  office." 
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KENDALL  COEFFICIENT  OF  CONCORDANCE. 


W  FORMULAE 
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Kendall  U  Statistic 


1 / 1 2  k 2 (N*-N) 

where : 

S  ■  sum  of  squares  of  the  observed 
deviations  from  the  mean  of  R; 

k  -  number  of  sets  of  rankings; 

N  ■  number  of  items  ranked. 

Chi-square  statistic 

XJ -  k (N- 1 ) W 
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